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Introducing the Curve Fitting Pac 


The HP 82484A Curve Fitting Pac is a powerful tool that enables you to perform functions that are not 
common to portable computing devices. The Curve Fitting Pac permits you to: 
e Fit a general model function (linear or non-linear) to a set of data using the (FIT program. 


e Determine local maxima and minima of a large claes of real-valued functions using the 
QOFTIMIZE program. 


The Curve Fitting Pac allows you to quickly and easily choose a model, fit a curve to your data, choose 


another model, and fit the curve again—all in a matter of.seconds. 


Features of the pac include: 
e A matrix editor that makes entering and editing data easy. 
e A built-in library of commonly-used fit models. 
e The ability to store data and then retrieve it for later use. 


e The ability to direct all intermediate and final results to a peripheral printer. 
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How to Use This Manual 


The information in this manual assumes that you have read sections 1, 6, 8, 9, and 11-14 in your HP-71 
Owner’s Manual and that you are familiar with the following HP-71 operations: keyboard operation, 
file operations, writing and running simple subprograms, manipulating data, using flags, and correcting 
error conditions. In addition, if you plan to use a printer with this pac, you should know how to install 
‘and use a printer using the HP 82401A HP-IL Interface. 


This manual is both a learning and reference tool. Read through section 1; “Getting Started,” for an 
_ overview of what the Curve Fitting Pac can do. Then read section 2, “Curve Fitting,” or section 3, 
“Optimizing a Function,” depending on which application you want to use first. Later, if you need 
. descriptions or reminders on how parts of the programs work, you can use these sections for reference. 
At the end of section 2, you will find “A CF 1T Example.” This is a comprehensive example for you to 
- key in to get familiar with the main program features. 


There are also several appendixes for your reference: 


e Appendix A, “Owner’s Information,” includes warranty and service information. 


e Appendix B is “Error and Status Messages.” If CF IT or OF TIMIZE cannot carry out an opera- 
tion, an error message will be generated. Refer to this appendix for an explanation of the messages 
the Curve Fitting Pac can generate. 


e Appendix C, “Numerical Methods,” describes the mathematics used in this pac and discusses situa- 
tions that are difficult for the program to handle. 


e Appendix D is “User-Accessible Routines.” Here you will find the subprograms in the pac that you 
can access and how they interface with the main programs and each other. 


e Appendix E is “Library Subprograms.” [=F 1T provides a library of model subprograms for some 
commonly used models. Refer to this appendix for a list of these subprograms. 


e Appendix F, “Applications File Format (HPAF),” describes the special format that this HP-71 
application uses to store fit data. 


e Appendix G is “Creating Your Own Model or Function Subprogram.” You’ll find information on 
syntax, calling relationships, and memory requirements, plus two examples of typical user-written 
subprograms here. 


e Appendix H is a list of the file names used by the programs in this pac. 


e Appendix I is a short glossary of terms. used in this manual. 


At the end of the manual is a complete subject index. 
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Section 1 


Getting Started 


Installing and Removing the Curve Fitting Module 


The curve fitting module can be plugged into any of the four ports on the front edge of the HP-71. 


CAUTIONS 


¢ Be sure to turn off the HP-71 (press (f] [ON]}) before installing or removing any module. 


¢ Whenever you remove a module to make a port available for another module, be sure to turn the 
HP-71 on and then off while the port is empty before installing the new module. 


© Do not place fingers, tools, or other foreign objects into any of the ports. Such actions could result 
in minor electrical shock hazard and interference with pacemaker devices worn by some persons. 
Damage to port contacts and internal circuitry could also result. 





To insert the curve fitting module, orient it so that the label is 
right-side up, hold the HP-71 with the keyboard facing up, and 
push in the module until it snaps into place. During this operation 
be sure to observe the precautions described above. 





To remove the module, use your fingernails to grasp the lip on the bottom of the front edge of the 
module and pull the module straight out of the port. Install a blank module in the port to protect the 
contacts inside. 
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What the Curve Fitting Pac Does 


The Curve Fitting Pac provides two main capabilities: 


¢ The ability to fit a general model function (linear or non-linear) with up to 20 unknown parameters | 
to a set of data. 


¢ The ability to determine local minima (or maxima) of a large class of real-valued functions with up 
to 20 variables. 


Both of these applications are based on the implementation of a powerful algorithm introduced by R. 
Fletcher and M.J.D. Powell. The method is known as the Fletcher-Powell Method (hereafter referred to 
as the FP Method) and is explained in detail in appendix C. 


A printer or a video display is not required to use this pac since all data and output can be directed to 
the HP-71 display. You probably will find a printer or video display useful, however, for viewing large 
amounts of stored data as well as intermediate and final results. If you have an HP 82401A HP-IL 
Interface and a compatible printer or video display, the programs will print (or display) the output in 
an easily understood format. If you want a printer or video display and don’t have one, contact your 
authorized Hewlett-Packard dealer for information on printers, video displays, and the HP 82401A 
HP-IL Interface. 


Using OF it 


Overview 


When you run the program (FIT, there are six main steps to go through to fit a curve to your data: 
. Create a model subprogram (or select one from the built-in library of model subprograms). 

. Enter the data points and optionally save them in a file. 

. Specify the name and location of your model subprogram. 


. Specify an initial guess for the model parameters. 


oo & &©O NY 


. Optionally do one or more of the following: 
© Compute the Chi Square value using the current model parameters. 
¢ Edit the model parameters. 

_ @ Edit the program control values. 

6. Fit the curve. 
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Flow Chart of =F IT Menus 


What follows is a flow chart of the steps involved in and the menus associated with the curve fitting 
procedure. 
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(-F IT Example: Stock Predicting 
Without worrying too much about the meaning of all the keystrokes, key in this example and see how 
easily CF IT fits a curve to a set of data. What you will do in this example is: 
1. Enter a set of data into a working array. 
. Enter an additional row of data to be evaluated but not used for the fit. 


2 

3. Save the data in a file. 

4. Fit the curve and get the results. 
5 


. Evaluate the model to predict the value of the unknown dependent variable corresponding to the 
data entered in step 2. 


Example: The asking price for three stock issues was recorded at the close of trading on six successive 
Fridays and is given in the following table. Assuming the linear relationship Y = aA + bB + c between 
the stock prices, what value can you expect for the price Y when A = 42.5 and B = 28? 


et [aa set | Sune 






* This row is the interpolate row. The weight of “Inf” causes this 
row to be ignored in the fit process. 


Input/Result 


RUN CFIT 


flats Edit Fit Suit? The main menu. 





You need to enter data from the keyboard, so 
press (D] to access the Data menu. 


Kha Load Sawe Print? The Data menu. 





Press {K] to select keyboard entry. 
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Input/Result 


Ho. of indpt varstTl 





= In this example there are two independent vari- 
ables (A and B). 





Ho. of data pointe? 
? There are seven data points, including the 
interpolate. 


iu 


The program branches to the editor and displays 
the first element of the first row of the array. All 
the values in the array default to 0 (except the 
weights, which default to 1). 


Enter the correct value of the first element from 
the table—stock A, week 1. 


The next element of the same row of the array is 
displayed. You can accept the default by pressing 
(END LINE], or you can enter another value. 


Enter the value of the second element from the 
table—stock B, week 1. 


The program steps through the array one element 
at a time. 


Enter the value of the third element from the ta- 
ble—stock Y, week 1. 


The last element in this row of the array (the 
weight) is displayed for editing. 


Press to select the default weight of 1. 


The first element in the second row of the array is 
displayed. 
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Input/Result 
24 


& , 5 [END LINE 


fu 


6.2 [END LINE 











Prat 
eer 





(9] inf [END LINE} 


Enter the value for stock A, week 2. 


The program will continue to step through each 
element of the entire array. 


Enter the value for stock B, week 2. 


Enter the value for stock Y, week 2. 


The weight for the second row is displayed. 


Use the default weight of 1. Continue to enter the 
values from the table as the program calls for 
them. 


The program prompts for the first element in the 
last row (the interpolate row). 


The value of stock A to be used to solve for the 
unknown value of stock Y. 


The value of stock B to be used to solve for the 
unknown value of stock Y. 


This is the unknown value we are looking for. 


Let it default to zero. 


The weight for the interpolate is displayed. 


While holding down the (9] key, type in inf. 
This gives this row a weight of infinity, thus effec- 
tively eliminating it from the fit process. 


Input/Result 


Cate Edit Fit uit? 


Ebead Load Say Frint? 


nane TE 





STOCE (END LINE 


Saywird... 
Tata Edit Fit Quit? 


SUBPPoOgr am rane 2H 


LTH (END LINE] 





FITL IB (END LINE} 
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The program has gone through the entire array 
and wraps back to the first element. 


Press this key to exit the editor and return to the 
main menu. 


Select the [ats option to display the Data 
menu. 


Press (S] to start the process for saving data to a 
file. 


The program prompts for the file name to save 
the data into. 

Save it into a file called STOCK. 

The program displays a message while the file is 


being created and the data copied to it. Then the 
main menu reappears. 


Press {F] to start the fitting process. 

The program prompts for the name of the model 
subprogram. 

LTH is the built-in linear model subprogram. 
Now (FIT asks for the name of the file that the 
subprogram is in. 


LIHW is in the library file FITLIE. 
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Input/Result 


Hin, 





2 [END LINE] 





Edit 








of Model Farms 





Eomtrols tye 


report 


CYHG? 


The model has three parameters (a, b, and c). 


Asks for a guess for the value of the first 
parameter. 


Since you probably have no idea, use the default 
of 0. Do the same for the other parameters. 


After the last guess is entered, the program dis- 
plays the Fit menu. 


Press [F] to fit the curve. 


For this example you do not need to edit the pro- 
gram controls; you can use the defaults. 


The program asks if you want to view an itera- 
tion-by-iteration progress report. 


Press ([N] for no. 





Input/Result 
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Assuming you don’t have a printer or video dis- 
play attached, the norm of the gradient at each 
iteration is displayed. (If you do have a printer or 
video display attached, the results are output as 
shown in the example in section 2, On pages 
47-49.) 


After the program converges, press to 
continue and see the results. 


Press to pause between each result. This al- 
lows you to scroll through the results at a 
comfortable pace. Just press when you 


want to go on to the next one. 


As you scroll through the results, notice that 
Pols, Pees, and Fé 23% give you the values for 
a, b, and c in your linear model, G¢ 1 1, GO23, 
and (3° 22> are the partials of Chi Square with re- 
spect to the model parameters. The other 
information will be described later. 


Since your ultimate goal is to get the predicted 
value for stock Y, press [M] to begin evaluating the 
model at your interpolate. 


The program asks if you want to evaluate one row 
or all rows. 


Evaluate row 7, the row with your unknown stock 
price. 
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Input/Result 


Row 7: PrP G47 oo95o9gaq The model predicts 77.85 for stock Y. 





(END LINE} 


Ceq Mdl Pems Fit 





Exit the curve fitting routine. 


Data Edit Fit Quit? 





(a) Quit the program. 


So, assuming a linear relationship Y = aA + bB + c between stock prices, and given a price for stock 
A of 42.5 and stock B of 28, you can predict the price of stock Y to be 77.85, or 77%. 


Using @FTINTSE 


Overview 
When you run the GF TIMIZE program to find local minima or local maxima, you go through a pro- 
cess similar to that for CF IT. You must: 

1. Create a subprogram for the function you want to optimize. 

2. Specify the name and location of the function subprogram. 

3. Make an initial guess for the variables. 

4, Test your guess (optional). 

5. Optimize the function. 


Flow Chart of GFTIMIZE Menus 


What follows is a flow chart of the steps involved in and the menus associated with the optimizing 
procedure. 
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RUN OPTIMIZE 


Subprogram name? 











File name? 


How many variables? 
vOLI we 2 


OPTIMIZE MENU 


Test Edit Opt Quit? 


Feddd Edit Controls ¢¥/N9? 





¥ 









MIN or MAXimize? 
Bound estimate? 


Approx. grad ¢Y/N9? 


Y 


Const. or Percent (CP) 


P 


Percentage? 


Gradient Limit? 


Line search tries? 


Q 
Progress report (YNQ)? 
N Y 


Pause on result ¢¥en3? 





Calculation 


CONVERGED ITERATION LIMIT 
Pause on results<¥-N>? Pause on results(t-H>? 
Final report Final report 


More iterations (¥NI? 
N Y 
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OFTIMNIZE Example: A Big Box 


Key in this OF TIMIZE example in the same way that you did for the ¢F I T example. In running this 
example, you will go through the five steps as outlined above. 


Example: You are designing a box that will be used to mail widgets. You want the box to have dimen- 
sions that yield the largest volume while still being acceptable to your local carrier. The postal restric- 
tions stipulate that the sum of the length and girth (perimeter of cross section) cannot exceed 100 
centimeters. What is the maximum volume for your box and what are the dimensions? 


Ww 


Considering the postal restrictions and since you have maximum volume when the length and girth 
sum to 100, you can use these equations: 


L + (2W + 2H) < 100 
V = WHL 


V(W,H) = WH(100 — 2H — 2W) 
100WH — 2WH? — 2W?H 


i 


All dimensions must be greater than 0, so you can impose the additional constraints 


0<W+H<50,W>0,H>0 
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First, type EDIT EXAMPLES and press (END LINE], and then enter the following subprogram called 
ECs and store it in a file named EXAMFLES. (For information on writing and storing subprograms, 
review section 12, “Subprograms and User-Defined Functions,” in your HP-71 Owner’s Manual.) 


= 
Tl 


SUB BOMCROs, Ges, t ,el 
N=PC1> @ H=PC2) @ BEEP 
VEWEHEC 18G-24N-24H 3 

2eHECSG-H-2EMS 
GOZ3=24WeCSA-W-2HH) 
EHO SUE 





a aes 
fad th 





TL ead ie 
me 
Nog 


— 
Pim] 


Now enter the following keystrokes and see how easy OPTIMIZE is to use. 











Input/Result 
RUM OPTIMIZE Run the program. 
Subprogram frame? ll The program prompts for the name of your 
subprogram. 
BOS (END LINE] 
File name? Prompts for the file where the subprogram is 
found. 
EXAMPLES 
How mang wa lables?s OF TIMIZE prompts for the number of variables 
in your function. 
Use the default of 2. Line 30 of the subprogram 


EO shows that W and H are the two variables. 


Weise The program prompts for an initial guess for the 

value of W. Line 20 of the subprogram identifies 

W as the first variable, ''¢ 1%, and H as the sec- 

ond variable, }'¢ 2°, 

= You don’t know what the value is, but go ahead 
and guess 5. 





Prompts for a guess for H. 





& (END LINE Guess 6 for H. 
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Input/Result 


Edit Opt Quit? After you enter the last guess, OPTIMIZE dis- 
plays the Optimize menu. 





Press (1) to evaluate V(5,6). Evaluating the func- 
tion at the current guess gives you the associated 
value of the function at that point. 





Return to the Optimize menu to prepare to enter 
the optimization routine. 


Test Edit Opt uit? 
Select the optimization routine. 
Edit Controls (YeHo? Asks if you want to edit the controls. 


Yes. You must edit the controls for this problem 
and all maximization problems because the pro- 
gram defaults to minimization. 


MIH or HAXimize FMI Asks if you want to minimize or maximize (the 
default is to minimize). 





MAS 
Bound estimate Ta Prompts for the bound estimate. 
It’s at least 2340, so guess 3000. 


Approx, grad (Yoho? The program asks if you want to have the gradi- 
ent approximated for you. 





Lines 40 and 50 in your subprogram do this for 
you, so you don’t need the program to do it. 
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Input/Result 


Gradient 





Use the default limit to stop iterating. 


Line se¢arch tries?ia 





Use the default. 


Ilterations?: 





Use the default to set the upper limit on the num- 
ber of iterations. 


Frogress peport CYHO>? The program asks if you want a progress report 
for each iteration ({Y] or [N]) or if you want to 
return to the Optimize menu ((Q)}). 


When you press or (N] the program starts it- 
erating. You hear a beep every time the sub- 
program is accessed from the main program. 


Es] 


At this point, you see the norm of the gradient for 
each iteration followed by the COHWERGED 
message, displayed when the program converges. 


1 [GRO 
2 IGROI: 
= IGEOL: 
4+ IGEOL: 
LOMVERGED 


et a 
Sp 
Lit 


+ + 
Se 
( 
tap ee Re ee 


Lo om mt was 


be pe ee ft 

tal io om] 

Ca oe fe 

mmmm 
o 


| 
i i 
me 
na! 


Pe tet 
eo tl ta 





END LINE Press [END LINE] to continue. 


The program asks if you want to pause between 
each result. 





Press so you can scroll through the results at a 
comfortable pace using [ENDLINE], assuming you 
don’t have a printer or display monitor attached. 
(If you do have a printer or video display at- 
tached, you won’t see this prompt. Printed results 
for this problem are shown on page 61.) 
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Input/Result 
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at 


at Wma a ha 
Wo mm ha 


mi Ee Et 
hee f Oo] 


DtoL + 4 
ima EE 
Fam 


ZEEE 
SSE 
BFE 
BSE 
44E 
44E 
it’ 


i 


Scroll through the output by pressing 
after each result. F «1 is the maximum volume, 
i Grd is the norm of the gradient (used in the 
determination by the program to stop iterating), 
Weis is the value of W, {=> is the value of H, 
Gi? and G“ 2% are the partials of V with re- 
spect to W and H. After the last item is displayed, 
you return to the Optimize menu. 


Press (Q] to quit the program. 


When you take the results for V, W and H, and then solve for L in the equation V = WHL, you have 
all the dimensions for your box. Your desired box size is 16.6667 by 16.6667 by 33.3333 centimeters 


with a volume of 9259.259 cubic centimeters. 
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Curve Fitting 


Introduction 


The &F IT program allows you to enter your data into an array, store the array in a DATA file, retrieve 
the array from the file, and edit the data. Then, you can fit the data to a curve, examine the results, 
and optionally choose another model for another fit. Many common fit models reside in built-in library 
files, making specification of your model as simple as providing the name of the subprogram and the 
name of the built-in library file in which it resides. (Refer to appendix E for information on the library 
files. If you find that you need to write the model subprogram yourself, refer to appendix G, “Creating 
Your Own Model or Function Subprogram.”) 


CF IT permits you to fit a model function (specified in a subprogram) to a set of data. The program 
uses the FP Method* to minimize the Chi Square function associated with your model and the data set 
(for details of the FP method, refer to appendix C, page 75). The number of independent variables (n) 
and the number of data points in your data set (m) is limited only by the amount of available memory 
in your system. The number of parameters (k) in your model is limited to 20. 


The model function you specify is represented by F = F(X(),P()) where X() is a vector of length n 
and P() is a vector of length k for which there are k unknown parameters. For example, the model 
function might be a simple third-order polynomial. In this case, you can represent the unknown param- 
eters (coefficients) by a vector of length 4 and the vector of independent variables reduces to a simple 
scalar value (in this case there is only 1 independent variable). 


Model: Py + P2X + P3X? + P,X3 
Fo = FON,POo) 
= Pelo + Peesee + POSSEROS + PO4a geo SR 
If you make 12 observations (collect 12 data points) for the above example, your data array will be 12 


by 3 (with 3 being derived from n + 2), and the 4 parameters to be determined are the coefficients of 
the polynomial model. 


* As mentioned in section 1, the method used is the “Fletcher-Powell Method,” abbreviated here and throughout the remainder of 
the manual as the “FP Method.” 
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Running the Curve Fit Program 
When you first run CF IT, by typing FU CF IT (ENDLINE], the main menu is displayed: 
Data Edit Fit Quit? 


The curve fit program is divided into three major parts. They are accessed from the main menu and are 
entitled Data, Edit, and Fit. The other option available in the main menu (f11i t) allows you to 
quit the program. 


Note: If you exit the program by any means other than the @ui t option, certain system functions 
are not maintained (for example, flag settings, rounding mode, and option base). 


The [ata part of the CF IT program contains commands for entering data points from the keyboard 
or from a file, saving entered data to a file, and printing the data array. The Edit part contains an 
editor for examining or modifying the data points. The Fit part contains the procedure for curve 
fitting. Each part is called from and returns to the main menu. The material that follows describes 
each of the three parts. 


Working With © 


{14+ allows you to enter data points from either the keyboard or a data file. You can also save the 
data points in a file and, using the Fr irt option, print the data points. 





‘From the main menu you press [D] to display the Data menu: 
Ebd Losd Save Print? 
All of the (sts options are selected from this menu. 


If you don’t want to use any of the [at 4 options, you can press (Q] to return to the main menu. 
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The Data Format 


The data items consist of n independent variables, a dependent value, and a weight signifying the 
confidence level. These items are stored in an array of m data points (rows) and n + 2 columns. 


For example, this 4 by 8 array represents a data set with 4 observations and 6 independent variables. 
Xi1 Xi2 Xig Xiz Xis Xie Yi7z Wis 
Xi X22 X23 Xeq Xo5 X26 Yo7 Woe 


’ 9 


X31 X32 X33 X34 X35 X36 Ya7 We 
X4qi X40 X43 Xg4 Xa5 Xge Yar Was 


Giving Weights to Data 


When (FIT creates a data array, each row has an associated column for the relative weight of that 
data. Data points given relatively small weights are considered “reliable,” while those given relatively 
large weights are considered less reliable. When you give weights to data points, the value of the 
weights should be equal to the standard deviations of the dependent variables. 


A reasonable approach to weighting is to use any reliable information you have regarding the true 
standard deviation as the weight. In the absence of any such information, use 1. 


Note: Do not use 0 as a weight. The Chi Square calculation involves a division by the weight. 
Using 0 results in a math error producing either «z=ro or 4.4 as an error message. 


Entering Data From the Keyboard (Eb) 


If you want to enter a new set of data from the keyboard, follow the instructions below. This set of 
instructions, as well as other sets in this manual, follow a format that gives the step number, the 
display you will see, and the instructions on what to do to complete the step. 


Step Display Instructions 
1 Kbd Load Save Print? Press [K] to start the process of creating a data set from 
the keyboard. 
2 Ho. of indpt vars? Enter the number of independent variables. Two col- 


umns will be added to this number to form the internal 
array: one for the dependent variable y; and one for the 
weighting factor w;. Any data already in an array will be 
destroyed when you create a new array. 


3 Ho. of dats points?il Enter the number of rows (data points) in the array. 


4 Bol, 1378 The program branches to the editor and displays the 
first element in the data array. 
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At this point the array has been created with all elements set to zero and the weights set to one. When 
you use the keyboard to enter the values of the elements, you simply edit their initial settings. The 
procedure for editing an element is shown in “Editing the Data,” starting on page 31. 


Loading Data From a File (L214) 


If the data already exists in a file, you can load the data into an array by following the procedure below. 


Step Display Instructions 
1 Kbd Load Save Print? Press (L] to load a data set from a file. 
2 Clear data YH? If you have data in the array, the L2ad option can ap- 


pend new data items to the existing set. If you wish to 
append the new data to the array, press [N]. To clear out 
the array, press [Y]. If no data is in the array when the 
Load option is selected, this question will not appear. 

3 LOAD: File naneti Enter the name of the data file. The file must be format- 
ted as an HPAF file. For detailed information on the 
HPAF format, refer to appendix F. If a mass storage de- 
vice is used, the file name must include the device 
specification (for example FOIHTS: TAFE). 

4 Loading... The program loads the file and then returns to the main 
menu. 


Saving Data to a File (342) 


Once the data is entered, it is often a good idea to save it to a file for future use. Instructions on how to 
do this come next. 


Step Display Instructions 
1 Ebd Load Save Print? Press (S] to save the current array of data in an HPAF 
file. 
2 SAVE: File name? Enter the name of the file to write into. The file may 


reside either in RAM or on a mass storage device. If a 
mass storage device is used, the file name must include 
the device specifier (for example “FOIHMTS: TAPE*). 

The program will create an HPAF file with the data in 
it. 


3 Qverwrite File ¢YeHs? If a file already exists with the name you supplied, the 
program will ask if you wish to overwrite it. If you press 
[N], the program will again prompt for the file name. If 
you press [Y], the program overwrites the previous file. 


4 Saving... The program saves the data and then returns to the 
main menu. 
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Printing the Data (Fr int) 


Once the data is entered, you can also get a printed copy of the data array (assuming you have a printer 
attached). The procedure below explains how. 


Step Display Instructions 
1 Ebd Load Save Print? Press (P] to print the data. The Curve Fitting Pac will 
send the data array to the current PE IHTER 1% 
device. 
2 Frinting... While printing is in progress, this message is displayed. 


If your HP-IL compatible printer is an 80-column printer, the output will be formatted as follows: 





Bab bee ee eet eet 
aos 2 8 8 


FAZBOOOE+O02 8. 500E-001 
This example is a printout of the data used in the comprehensive CF IT example starting on page 40. 


Note: If your printer prints using a narrow field, like the HP 82162A Thermal Printer, the printout 
_ will be formatted in easily read columns. 


After the data has been printed, the program returns to the main menu. 


Editing the Data 


Edit allows you to enter and modify arrays of data used by “F171. This part of CF IT is selected 
from the main menu by pressing (E]}. 


When you enter the array editor, the first display you see contains the value of the first element in the 
array: 

mad, ls=nann 
The display consists of three items: 


¢ A letter giving the type of element displayed: » for an independent variable, ‘* for a dependent 
variable, and || for a weight. 


e A pair of numbers in parentheses giving you the row and column address of the current element. 


e The current value of the element. 
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Editing an Element 
All of the HP-71 line editing features (such as (>], [<], (17R), [BACK], [-CHAR], and the command stack) 


are available in the array editor. If you are unfamiliar with how these features are used, refer to section 
1, “Getting Started,” in your HP-71 Owner’s Manual. 


Any time an element is visible it may be edited and the new value entered into the array. The element 
is edited by typing over the current value, and it is entered by pressing [ENDLINE]. All unassigned 
elements in a row are displayed as & except for the last one. The last element (the weighting for the 
observation) defaults to 1. 


When you edit an element, you can use both numbers and numeric expressions for the value of an 


cc = 


element. For instance, {© 1+50R¢25%2 3.2 is just as acceptable as = for an entered value. 


Moving Around the Array 


When the editor is running, a number of keys have been redefined to help you move about within the 
array and to help you insert or delete rows and columns. These keys are broken into the following 
groups: 


e The direction keys for moving through the array ((W], (DJ, (XJ, and [A)). 

e The command keys for manipulating columns and rows ({U], [0], [M], and [F)). 
e The endline direction keys ({S] in combination with the direction keys). 

e The quit key ((Q)). 


The following representation of the keyboard shows the keys that are redefined when you are in the 
array editor. 


GHeR0DTGoOR 


Column 


Qalalelolalolalele 


Endline Right 
Direction 


OGOUBAROOF 

own celui I 
N 
= 
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The Direction Keys. The direction keys are found in the cross on the keyboard diagram. You can use 
the direction keys in combination with the [f] key to move anywhere in the array. Once you get to the 
row and column you want, you can examine the contents of the element at that location and/or modify 
it. 


The (W}, (DJ, (x), and [A] keys are respectively the up, right, down, and left keys. These keys move you 
through the array one element at a time in the corresponding direction as shown in the following table. 





You can move to one of the boundaries of the array by pressing the [f] key and a direction key. You can 
think of the [#] as standing for the word “far.” 


¢ The (f] (0) and (f) (A) move to the far right and far left boundaries of the array. 
e The and move far up and far down to the boundaries of the array. 
All the direction keys will move across the boundaries to “wrap around” to the opposite side of the 


array. For example, in a 4 by 8 array, if you start at #1, 1 and press (DJ, you will goto Wei, 43 
(the far-right side of the array). Then, if you press (0), you will see #¢ 1,4 again. 


Note: If you are in the editor and want to enter an expression using letters that have been re- 
defined, you must hold down the (@] key while typing the letters. For example, since the key 
has been redefined as the G01T@ command, you can only enter a weight as in# by holding down 


(9} and typing inf. 
The Command Keys. The command keys can be associated with their gold, shifted functions on the 
HP-71 keyboard. The command keys and their functions are as follows: 
e The key selects the DEFine column command. 
e The (0] key selects the AOC command. 
¢ The (M] key selects the DELETE command. 
e The (F] key selects the GOTO command. 


The DEFine column command is used to assign values to the columns. It can be very useful when you 
want to enter data with a constant interval between points (for example, every year from 1954 to 1984 


. or 10-degree steps from 30 degrees to 120 degrees Fahrenheit). To do so: 
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1. Press (U)}, at which time the program prompts 


Col, Start, 


tep’ 


2. Enter the column to fill, the starting value, and the step size, separated by commas. After the 
information has been entered, the program returns to regular editing. 


Like all the keyboard commands, the define command can be used at any time while you are in the 
editor, regardless of where you are in the array. Also, if you get into this command accidently, press (Q]} 
to return to regular editing. 
The ACC command makes it easy to add a row or column to the array. To do so: 
1. Press [0], at which time the program prompts 
AQO: Rew or Column? 
2. Do one of the following: 
e Press [R] to add a row to the array; the program will display 
Add new row at ?m+7 


with one more than the total number of rows (data points) as the default address for the new 
TOW. 


e Press to add a column to the array; the program will display 
Add new column atta+7 
with one more than the total number of independent variables as the default address for the 
new column. 
e Press (Q] to exit the command. The program will return to regular editing. 


3. Enter the address of the new row or column. If the address coincides with an existing row or 
column, the array will open to create a space. The new row or column will be filled with default 


values. 


If you add a row or column at the address of an existing row or column, the existing row or column 
address (and all those rows or column addresses beyond it) increases by one. For example, if you enter 
a new row at row 3, the old row 3 becomes row 4, row 4 becomes row 5, and so on. 
The ('ELETE command is used to delete a row or column from the array. To do so: 
1. Press [M], at which time the program prompts 
DELETE: Row or Column? 
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2. Do one of the following: 
e Press [R] to delete a row from the array; the program will display 
Delete pow mumber tn 
with the current row as the default. 
e Press to delete a column from the array; the program will display 
Delete column rumber in 
with the current column as the default. 
e Press [Q] to exit the command. The program will return to regular editing. 
3. Enter the address of the row or column to delete. The program will display 
Delete row an tvYeHo 
or 
Delete colo oan ¢¥eHa? 
4. Press (Y} for yes to delete, or [N) for no to exit. 
After the deletion, the program returns to regular editing at the last displayed element or, if the 
corresponding row or column or a row or column before it was deleted, to an element near the pre- 
viously displayed element. If all rows or columns are deleted you will return to the main menu. 
The 201T0 command allows you to move directly to a specific element in the array. To do so: 
1. Press (F], at which time the program prompts 
Row, Col? 
2. Enter the row and column address of the element in the array. After the address is entered, the 


program displays the the element for review or edit. 


Endline Direction. After the key is pressed to enter an updated value into an array, the 
next array element is displayed for editing. The direction the program moves to display the next ele- 
ment is called the endline direction. The endline direction is set with the key, the key in the middle 
of the direction keys. 


The default direction is to the right, so that when is pressed the next element to the right is 
displayed. The first element in the next row is displayed after you enter the value in the right-most 
column. Using the default endline direction, you can easily input your data into a matrix by editing 


each element and pressing [END LINE]. 
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There are three possible endline directions: 
e To the right: This is set by pressing (S] for set endline direction followed by the right direction key, 
(D). 
e Down the columns: This is set by pressing (S] followed by the down direction key, [Xx]. 


e No motion: This is set by pressing (S] followed by (S] again. This will cause the same element to be 
displayed after pressing [END LINE]. 


When you press (S], the program will display 
Direction: O.8,5 or OF 


At this point, you can press (D], (X], or (S], depending on the endline direction you want. You can also 
press (Q] if you want to escape the command. When you press one of these endline direction keys, the 
program sets the endline direction and returns you to the last element displayed. The following table 
summarizes the effects of setting the endline direction. 


Direction Endline Current Element After 
Keys Direction | Element | Pressing Endline 


Right. 
Down. 
No motion. 






Exiting the Editor. To exit (quit) the array editor press the [Q] key. This will return you to the main 
“menu. This, too, can be used regardless of where you are in the array. 


Fitting the Curve 

Once your data has been entered and, optionally, saved in a file, you are ready to fit the curve. The 
Fit procedure involves: 

. Specifying the model. 

. Editing the parameters. 

. Optionally evaluating Chi Square with respect to the current model parameters. 

. Optionally evaluating the model at one or more points. 


. Optionally editing the program controls. 


a oOo fF WO NH 


. Performing the fit. 


Each of these steps will be described next. 
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Specifying the Model 


The first step in the process is to specify the model subprogram to the (F I T program. This is done as 
shown below: 


Step Display’ Instructions 
1 Data Edit Fit tuit? From the main menu, press (F] to begin curve fitting. 
2 Subprogram mane Til Enter the model subprogram name to be called by 
CFIT. 
3 File name? Enter the name of the file containing the subprogram. 
4 Ho. of Model Farms? Enter the number of parameters in the model. 


Every model, whether preprogrammed in the Curve Fitting Pac or written by you, has a given number 
of model parameters (k) and independent variables (n). Make sure that the number entered in step 4 
agrees with the actual number in your model subprogram. If the number entered is too small, you will 
probably get an error when the model is evaluated. If the number is too large, you will get incorrect 
results. 


Editing the Parameters 


After you enter the number of parameters in your model, the program prompts for the value of the first 
parameter in the initial guess. You can edit each parameter as outlined below: 


Step Display Instructions 
-1 Perio Enter the first parameter in the initial guess. The first 
time you run the program, the parameters default to 0. 
2 Peeo7e If there are more parameters to be entered, the program 
will prompt for them. Edit the remaining parameters as 
in step 1. 
3 Ceq Mdl Frms Fit uit? Once you have edited the last parameter, the program 


displays the Fit menu. 


Options From the Fit Menu 


Once you access the Fit menu, you can evaluate Chi Square at the current guess by pressing [C], eval- 
uate the model by pressing (M], edit the parameters by pressing [P], fit the curve by pressing (F], or 
quit and return to the main menu by pressing (Q]. The first three of these choices ((C], [M], and (P}) 
are optional within the curve fitting process. 


Evaluating Chi Square. If you press to evaluate Chi Square the program will display 
ChiSq=nnn 


You then press'(END LINE] to return to the Fit menu. 
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Evaluating the Model. To evaluate the model at any row (using current model parameters), follow 
the instructions detailed below. 


Step 
1 
2 


Display 


leq Mdl Perms Fit Quit? 


Row # Cor 


AlloTe 


Instructions 
Press [M] to evaluate the model. 


Enter the row to be evaluated, or enter to evaluate 
the model at all points. After the program evaluates the 
model at the appropriate row(s), the results will be 
printed or displayed and the program will return to the 
Fit menu. 


Editing the Program Controls. Before a curve is actually fit to your data, you can edit the program 
controls as shown below: 


Step 
1 
2 


6a 
- 6b 


Display 
Cseq Mal F 
Edit cant 


rms Fit uit? 


rales (Yeo? 


Min ChiSq estimateta 


APProax, 4g 


Const. or 


Gradient 


radeyy,No? 


FercentccoPa? 


Limit’, Aa1 


Instructions 
Press (F] to start the procedure for fitting the curve. 


Press if you want to edit the controls that affect the 
numerical calculation, or (N] if you want to proceed with 
the calculation using the current controls. If you enter 
(N] at this point, the procedure continues at step 10. 


If you have reason to believe the minimum Chi Square 
exceeds a given positive value, enter the value here to 
improve program performance. The default estimate is 
zero. 


If your model subprogram includes the gradient calcula- 
tion (all of the models in the library do), press (N] and 
pick up the procedure at step 7. If it’s necessary to 
approximate the gradient, press [Y]. (For information on 
the gradient and how it is used in this pac, refer to sec- 
tion 3, page 61.) 


Press (P] if you want to use a percentage of the param- 
eters for Delta. If you want to use a specific constant for 
Delta, press (C]. (If you are unfamiliar with how Delta is 
used to approximate the gradient, refer to appendix C, 
page 82 for details.) 


Enter the constant for Delta, or use the default value. 
Enter the percentage for Delta, or use the default value. 


Enter the gradient limit, or use the default. This limit, 
compared with the norm of the gradient at each iterate, 
is the criteria used to determine convergence. 
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Step Display Instructions 


~) 
ts! 


8 Line search tries?l Enter the maximum number of tries to be made in the 
line search routine. The default number of tries is 10. 
(For details on the line search routine, refer to appendix 


C, page 76.) 


"25 Enter the maximum number of iterations to be made in 
the attempt to converge. The calculation will normally 

converge in less than 25 iterations, so, for convenience, 
the default is 25. 


9 Iteratians 


Performing the Fit 


If you chose not to edit the program controls, or if you have completed editing them, you are ready to 
have the program perform the actual fit. CF IT makes its fit by finding the local minima of the Chi 
Square function. The “absolute best fit” may not be found. Instead, CF IT may converge to a “local 
best fit.” (This situation is most likely to occur when you’re using models containing periodic func- 
tions.) When the program prompts to determine if you want a progress report, it is ready to fit a curve 
to your data. 


Step Display Instructions 


10) Frogress report Hit? At each iteration, Chi Square, the gradient norm, the 
parameters, and the gradient of Chi Square can be 
output. If this information is desired, press [Y]. This 
information is sent to the current FRIHTEF I de- 
vice. The default is to display the iterate number and 
the gradient norm on the current DISPLAY I de- 
vice. If you get to this point and decide that you don’t 
want to go through with the calculation, press (Q] and 
the program will return to the Fit menu. 


If you don’t want a progress report, press [N]; the program will start iterating and you will see the 
display 


Be [Grdl: ananannE nnn 
For each iteration, the program will display the iterate number and the norm of the gradient. 


If you do want a progress report (if you press [Y]) and don’t have a printer attached, you will see the 
display 


Fause on resultetY-HoF 


EL! 
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If you press [Y] in response to this prompt, the program will stop between each result until is 
pressed. If you do not want to stop (if you press (NJ), you will see the progress report at the current 
CELA’ rate. Hither way, the program starts iterating at this point and, if you have a printer attached, 
the progress report will be printed. 


For an example of what the printed progress report looks like, refer to pages 47-49. 


-Getting the Results 


Once the program has completed the required iterations, it will have converged or it will have reached 
the iteration limit without converging. 


If the program did not converge, you will see the following: 


Step Display Instructions 
1 ITERATION LIMIT Press to go to the next display. 
2 More iterationscY Hi? Press to continue iterating (starting from the last 


iteration) with the same number of iterations as speci- 
fied in the program controls, or press [N] to print the 
results. 


Note: You may also see a numeric computation message, such as FIT ERR~~ 
Tries + Limit. If this happens, press [f] to return to the Fit menu. 


If the program did converge, you will see: 


Step Display Instructions 
1 COHVERGED ; Press to continue. 
2 Pause on results¢'<.N>? Press (¥] if you want to use to step through 


the results, or press (N] if you want the results output at 
the current [1ELA‘ rate. You will not see this prompt if 
you have a printer attached. 


As with the progress report, the final results will be sent to the current FF IHTER 1 device. To see 
an example of printed final results, refer to page 49. 


After the results have been printed, the program returns to the main menu. 


The material that follows is a comprehensive © F I T example for you to key in. This example uses many 
of the data entry, editing, and function evaluation features available in this pac. It also uses FL‘, one 
of the library subprograms available for ¢F 1T. At the end, a discussion is included on interpreting the 
results. 
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There are eight main steps to go through in this example. You will: 
. Enter data into a working array. 

. Save this data into an HPAF DATA file. 

. Supply the name of the model subprogram from the library. 
Supply control information. 

. Get the fit results. 

. Add another data point to the array for evaluation. 


. Evaluate the model at the extra point. 


SCI DA PR wO NY 


: Interpret the results. 


Setting Up the Problem 


Example: Suppose you have taken the data in the following table. 


1 
2 
3 
4 
5 
6 
7 
8 
9 





What are the coefficients of the fourth degree polynomial that best fit this data, and what is the value 
predicted by the model at X = 27? 


Notice that the weights have all been given a value of .85. In this example you can assume that the 
standard deviation of the dependent variable (Y) is .85 and that it does not vary as a function of the 
independent variable (X). 
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Entering the Data 

As the first step in this example, enter the data as follows: 
Input/Result 

RUM CFIT Run the program. 


Date Edit Fit Quit? The main menu. 





You need to enter the data... 


Kbd Load Save Print? 


from the keyboard. 








Ho. of indet vars? The program prompts for the number of indepen- 
dent variables in the problem. 
i There is only one dependent variable in this 
example, X. 
Ho. of data points? The program prompts for the number of data 
points (or rows) in the table. 
a There nine rows in the table. 


The program asks for the value of the first ele- 
ment in the first row. 





Enter the appropriate value for the element from 
the table. 


Asks for the value of the second element in the 
first row. 


Asks for the value of the third element in the first 
row. 


L685 Enter the weight for row 1. 





Input/Result 








24, ° (END LINE 
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Asks for the value of the first element in the sec- 
ond row. 


Enter the first value in the second row of the pre- 
vious table. 


Enter the second value in the second row. 


Enter the weight for row 2. Continue entering the 
data in the same manner for all the rows as the 
program prompts for the values. 


The program asks for the value of the first ele- 
ment in the last row. 


Asks for the second value in the last row. 


Asks for the weight for the last row. 


The editor wraps around from the last element in 
the array to the first element in the array. 


Press the [Q] key to exit the editor and return to 
the main menu. 
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Saving the Data 


The second step in this problem involves saving the data to a file. 


Input/Result 


CTlate Edit Fit tuit 





Kbd Load Save Print’? 





SAVE: File nameTil 





POLY OATA 


oe = 


Data Edit Fit Quit? 





Specifying the Model Subprogram 


Go to the Data menu. 


Save the data to a file named... 


FOL YOATA. 


After saving the data to a file, the program re- 
turns to the main menu. (A printout of the data 
used in this example is shown on page 31.) 


The third thing to do is to specify the model subprogram to CF IT. 


Input/Result 


(F) 


Subprogram mame Ti 





FOL‘ [END LINE 


Start the curve fitting process. 


Asks for the name of the model subprogram. 


Supply the name FUiL.° since your polynomial 
function is covered by this library subprogram. 
(FIL is the built-in model subprogram that 
handles all polynomials through degree 19.) 


Input/Result 


File 





hame sl 


FITLIE (ENDUNE] 






Ha, 








= [END LINE 





of Model Farms? 
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Asks for the file where the subprogram can be 
found. 


FOL‘ is found in the built-in library file 
FITLIE. 


Asks for the number of parameters. 


There are five coefficients in a fourth-degree 
polynomial. 


Asks for an initial guess for the first parameter. 
The program will go through all five parameters 
to allow you to supply an initial guess for each 
one. 


Enter the default value of 0 since you have no 
idea what the true value is. Do the same for the 
remaining parameters. 
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Editing the Controls 


Now you need to go through the process of editing the program controls. 


Input/Result 





Edit comtrols ¢YeHa? 





Min ChiSq estimate ?Ts 





Approx, grad CY¥eHa? 











lterations? 





The Fit menu. 


Press (F]. 


The program asks if you want to edit the controls. 


Yes. You need to edit the controls for this problem 
because the default gradient limit is unreasonable 
and, if used, eventually produces an error con- 
dition involving the gradient limit. 


Asks for the minimum Chi Square estimate. 


You have no idea, so use the default value. 

Asks if you want the program to approximate the 
gradient. 

No, because the subprogram calculates it. 

Asks for the gradient limit, which is the criteria 
used by the program to determine convergence. 
Enter 1.00 as the gradient limit. Do not use the 
default here; it is unrealistic for this problem. 
Asks for the limit on line search tries per 
iteration. 


Use the default. 


Asks for the limit on the number of iterations. 


Use the default. 
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Input/Result 


Asks if you want an iteration-by-iteration 
progress report showing intermediate results. 





Getting the Results 
When you see the previous display, you are ready to get results. 


Input/Result 


Press for yes to see a progress report. After 
you press [Y], assuming you have a printer at- 
tached, the program prints the results. If you 
don’t have a printer attached, the results will be 
displayed as in the example in section 1, starting 
on page 14. Assuming your printer is an 80-col- 
umn printer, your intermediate and final results 
will be formatted as follows: 


Initial values: 
Chi-square = 372820, 732564 


Gradient Norm = BABES1i0901. 56 


No. Farameter Gradient 
O, OOOOOOE+000 -—3. G66021E+005 
2 OO, OODQQDDE +000 ~1.181129E+005 
a 0, OOONQODE+000 —4. 455 9897E+006 
4 Oo. OOOQQODE+000 —-1.876959E4+008 


5 OY. QQQOQOQ0E+000 —-8. 3B1L408E+009 
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Iteration: 1 


Chi-square = 120455,210198 
Gradient Norm = 90420035, 83697 


No. Parameter Gradient 


ee mee meee teem sone meee meee erste conte seem evens coun cana omen veeet wena eeere seven wean 


2. 787581E-O11 ~1.878835E+003 
8.516494E-0109 -—3. 1801 350E4+004 
3. 2 l2909E-00g ~. 768235 2E+005 
1. S53375E~-006 —9, OF 1L261E+006 
6.043 386E-005 . BB OBZSSZLE+O0S 


CAE td hie 


BK 


Iterations 


Chi-square = 835282. 1526088 


Gradient Norm = 112017, 632547 


No. Farameter Gradient 


1 1.615609E-006 ~1. 1408S3E+002 
o 2. 7 S463S5E~-005 —1. 265068E+004- 
me 4. 960367E-004 ~1.110427E+005 
4 7. 7OB764E-O003 7. A4FSS91 E+O003 
ot ~1.95 16426-0004 —1, 008O49E+002 


Iterations 3 


Chi-square = 12502. 1022206 
Gradient Norm = 1074.98139404 


No. Farameter Gradient 
1. 305732E-002 ~1,887295E+002 
1.471068E-—-001 -1.O051068E+003 
1.271939E+000 1,233103E+002 
~S.8L27256-008 —-4, 131554E+000 
7. O94064E-004 3. FLlS990E-O0O1 


LAS Gi be 


Iteratian: 4 


Chi-square = 4051. 30071327 


Gradient Norm = 43. 402446567344 


No. Parameter Gradient 
1 3. 1O1776E +000 4, 272603E+001 
2 1.735852E+001 —7,,619956E+000 
a —-5. 776100E—-00]1 4, 4222 28E-001 
4 8. 998385E-004 ~9, SSSB30E-O038 
5 1.291139E-004 3. 4692280E-002 


Iteration: 5 
Chi-square = %.64992479158 


Gradient Norm = .85282104649542 


No. Farameter Gradient 
1 —-1. FFIBS7E+O002 -2,. B17993E-007 
2 5S. 747584E+oo1 —-1,082907E-005 
3 ~ 3. 270084E+000 —-4, 438699E-004 
4 7, 048645E~-002 —-1.910917E-002 
a ~4,780977E-004 -8.526168E-001 


Chi-square = 3.64992479158 
Percentage goodness of fit = 45.500 


Gradient Norm = .8528%1069542 


No. Farameter Gradient 
1 “1.791857E+002 2. B1799SE-007 
es 3. 747584E+o001 —-1.082907E-005 
a ~R. 2S7OONB4E+000 —~4, 4358699E-004 
4 7.048645E-002 ~1L.910917E-002 


o ~4. 78097 7E-004 ~8.526168E-001 
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For each iteration you are given 
e I—the current iteration (guess) number. 
e P,, ..., P5—the model parameters (or polynomial coefficients). 
e 0x’/dP,, ..., 0x’/AP;—the gradient vector at (P,, Po, ..., Ps). 
e ChiSq—the value of Chi Square at (P,, Po, ..., Ps). 
e |Grd|—the gradient norm (measures the flatness of Chi Square). 
Notice that after five iterations the Chi Square value was reduced from the initial value of 373,820.73 tc 


3.64992479158 (where convergence occurred). The final coefficients make the fourth degree polynomial 
look like this: 


F(X) = Py + P.X + P3X2 + P,X3 + P,X4 


where 
P, = —1.791857 E2 
P, = 5.747584 E1 
P,; = —3.270084 E0 
Py = 7.048645 E—2 
P; = —4,.780977 E—4 


The following graph shows the nine data points and the polynomial function determined above. 


350 


268 
1S 


166 
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With the results, you also get something called percentage goodness of fit, which has the value 45.50. 
This number can be used in the interpretation of the results, which is discussed after the model 


evaluation. 


Evaluating the Model 


Now that you have the results of the fit, continue on with the example by adding a row and then 
evaluating the model at X = 27 to answer the original question: “What is the value predicted by the 


model at X = 27?”. 


Input/Result 


Req Mdl Peme Fit Quit? 


Data Edit Fit uit? 


ADO: Row or Column? 


Aidd new row at?id 





After printing the results, the program returns to 
the Fit menu. 


Press [Q] to return to the main menu. 


Press [E] to access the data editor. 


The program enters the array at the first element. 


The letter O has been redefined as the ALL com- 
mand. (Refer to page 32 for a description of the 
redefined keys.) 


Asks if you want to add a row or column. 


Add a row. 


Asks where to add the new row. The program de- 
faults to one more than the current number of 
rows. 


Use the default to make the new row number 10. 
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Input/Result 


Meo rk ing... A new row is added to the array with default 
HCL, 13=8 values. 








Use the default value since this number is not 
used in the evaluation. 


The default weight. 


While holding the [9] key down, type inf. This 
gives this row a weight of infinity. With this 
weight, the row will not be considered in the curve 
fitting process. 


Exit the editor and return to the main menu. 


Data Edit Fit uit? 


SUbPrOoOgr am name? POLY Access the Fit menu by scrolling through the sub- 
File mame TFITLIB program and parameter prompts using the 
Ho, of Pode] Par hee key. 

Pcoqjo?-173, 
Pea? 
Fr: ‘f 
Foqae 
Peas 


1 tee 
Ty fa 


mo m0 
mm 


a 





mim) 


I fe mY 


—pom fae! 
4k 


J iT) 
I 


a) 
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Input/Result 


leq Mdl Perms Fit Quit? 


Press [M] to start the process for evaluating the 
model. 


Row # or ALlLOTH The program asks for the row number to be eval- 
uated (or if all rows are to be evaluated). 





Row 10. 


The function value at X = 27. 





END LINE Press [END LINE] to return to the Fit menu. 


leq Mdl Perms Fit Quit? 


Cate Edit Fit Guit’? 





(a) 


Interpreting the Results 


An important point about the program is that achieving convergence does not necessarily mean that the 
model you have chosen is appropriate. ¢F 1 T merely tries to come up with the best solution for the 
model you have chosen. You can see from the graph on page 50 that the model chosen for this example 
is a good one. However, in the general case, where graphs may be inappropriate, statistics, specifically 
the Chi Square value, can be used to determine the acceptability of the model. 


Assuming the dependent variables are normally distributed with standard deviations equal to the 
weights, and assuming the model fits your data well, the value identified by the program as Chi Square 
will be x7(v) distributed. The degrees of freedom (the number of data points minus the number of 
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parameters) is v. (The mean value of a x(v) distributed variable is v.) In this example the Chi Square 
value is 3.6499 and there are 4 degrees of freedom (9 data points minus 5 parameters). In general, if the 
above assumptions are satisfied, you can expect Chi Square to be close to the mean value ». 


Assuming the weights are valid, an unreasonably large value for Chi Square probably means that the 
model selected is inappropriate for the data. 


The determination of “unreasonable” is usually made beforehand based on how much risk you are 
willing to take of rejecting a model when it is truly appropriate. For example, assume you are willing to 
take a risk of 10% of rejecting a valid model based on the previous criteria. You would see from Chi 
Square tables that 90% of the time a x7(4) (Chi Square with 4 degrees of freedom) distributed variable 
will be less than 7.78. Consequently, you would reject the model if the computed value exceeded this. 
Since 3.6499 is less than 7.78, you cannot reject the model for this example on that basis. In other 
words, the model for this example is reasonable. 


Because of the utility of the Chi Square statistic in evaluating the validity of the model, the BASIC 
subprogram FCEHTCHI has been built into the Curve Fitting Pac. This program eliminates the need 
to refer to Chi Square tables. FTENTCHI accepts values for U, the upper limit, and V, the degrees of 
freedom, and returns P, the probability that a Chi Square distributed variable is less than U. 


CFIT calls PFCEHTCHI, passing it U (the value identified by CF IT as Chi Square) and V (the num- 
ber of data points minus the number of model parameters) and obtaining in return the probability, P. 

The value identified by CF IT as the percentage goodness of fit is 100 x (1 — P). This is the percent- 

age of time that a Chi Square distributed variable with V degrees of freedom would exceed the (FIT 

value. If the percentage goodness of fit is less than your acceptable risk percentage (in this example, 

- 10%), you would reject the model. Since the 45.50 returned by the program in this example is greater 

than 10, you have no reason to reject the model at the 10% significance level. 
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Optimizing a Function 


Introduction 


Optimization is a term used to describe a class of problems in which the objective is to find the mini- 
mum or maximum value of a specified function. Often, the interest is focused on the behavior of the 
function in a particular region. Thus the goal becomes one of finding a local minimum or maximum. 
The OF TIMIZE program uses the FP Method to determine local minima or maxima for real-valued 
functions whose gradient vectors can be defined analytically at each point. The functions can have up 
to 20 variables. 

Remember, the process you go through in OPTIMIZE involves: 

1. Creating a subprogram for the function you want to optimize. 


2. Running GF TIMIZE and specifying the name and location of your function subprogram to 
OPTIMIZE. 


3. Making an initial guess for the variables. 

4. Optionally: 
© Evaluating your function at the current variables. 
® Editing the current variables. 
¢ Editing the program control values. 


5. Optimizing the function. 


Creating the Function Subprogram 


Before you run the program, you need to create a subprogram for your function and have it in memory. 
For information on creating the required subprogram, refer to appendix G, “Creating Your Own Model 
or Function Subprogram.” 





Running the [: . Program 


GFTIMIZE may be invoked through either a FLIH command or a CALL command from another 
program. 


You run OPTIMIZE by typing RUN OPTIMIZE (ENDLINE]. 
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Specifying the Subprogram 


Specifying the name and location of your function subprogram is the first step in OF TIMIZE. The set 
of instructions that follow, as well as other sets in this section, are formatted to give the step number, 
the display you will see, and the instructions on what to do to complete the step. 


Step Display Instructions 
1 Subprogram name? il When you run OF TIMIZE, this is the first display you 
see. Enter the name of the function subprogram to be 
called. 
2 File name? Enter the name of the file containing the subprogram. 


Enter the number of independent variables (the default 
is =). 


fat 


3 How ward vaerisakl 


Te 
tip 


Make sure that your entry for the number of variables agrees with the number of variables actually in 
your function subprogram. If the number entered is too small, you will probably get an error when the 
function is evaluated. If the number is too large, you will get incorrect results. 


Editing the Variables 


After you enter the number of variables, the program displays the first variable for editing. You can 
enter (or edit) the values by stepping through each variable. As in © F IT, all the features of line editing 
in the HP-71 are available for use. (For more information on the keys used in line editing, refer to 
“Keyboard Operation” in section 1 of your HP-71 Owner’s Manual.) 


Step Display Instructions 
1 Welaee Enter the value of the first variable in the initial guess. 
Notice that the variables in the first guess default to 
zero. 
2 Wego? Enter the second variable. The program will prompt for 


the remaining variables in a similar manner. When you 
enter the last variable, the program will display the Op- 
timize menu. 


Options From the Optimize Menu 
This is the Optimize menu: 
Test Edit Opt uit 


From it you can evaluate (test) your function at the current guess, edit the variables again, optimize 
(minimize or maximize) the function, or quit the program. 


Testing the Function 
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To test the function using the current variables, follow the steps outlined below. 


Step Display 


1 
2 


Test 


Edit 


F=nnn 


Opt uit? 


Editing the Controls 


Instructions 


Press [T]. 


The program evaluates the function at the current vari- 
ables and displays the result. Press to return 


to the Optimize menu. 


Once you have specified your subprogram, entered your variable values, and optionally tested your 
function, you are ready to optimize. Before you actually start the optimization, though, you have the 
option of editing the controls that affect the numerical calculations. The steps to do this are shown 


next. 


Step 
1 
2 


Va 
Tb 


Display 


Test 
Edi t 


MI 


B 


APerox, 


Const. or 


urd 


Edit 


Opt uit? 


comtrols (Yea? 


or WAKi mize Shih 


Bstimate Te 


Tt 


grad CYeHo? | 


Percentccrpi? 


Instructions 


Press [0]. 


Press if you want to edit the controls that affect the 
numerical calculation, or (N] if you want to proceed with 
the calculation using the current controls. If you enter 
{N]} at this point, the procedure continues at step 11. 


Enter 1H if you want to minimize the*function, or 
MAS if you want to maximize it. The default is to mini- 
mize it. 

Enter the bound estimate (lower bound for minimizing 
and upper bound for maximizing). A good estimate of 
the bound can help the program to converge sooner than 
it otherwise would. The default is to &. 


If your subprogram includes the gradient calculation, 
press (N] and pick up the procedure at step 8. If you 
want the gradient approximated automatically, press [Y]. 


Press [P] if you want to use a percentage of the param- 
eters for Delta. If you want to use a specific constant for 
Delta, press [C]. (Refer to appendix C for details on how 
Delta is used to approximate the gradient.) 


Enter the constant for Delta, or use the default value. 
Enter the percentage for Delta, or use the default value. 


Enter the gradient limit, or use the default. This limit, 
compared with the norm of the gradient at each iterate, 
is the criterion used to determine convergence. 
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Step Display Instructions 
9 Line search triesT1la Enter the maximum number of tries to be made in the 
line search routine. The default number of tries is 10. 
(For details on the line search routine, refer to appendix 
D.) 
10 Iterations ?23 Enter the maximum number of iterations to be made in 
the attempt to converge. The calculation will normally 


converge in less than 25 iterations, so, for convenience, 
the default is 25. 


Performing the Optimization 


If you chose not to edit the program controls, or if you completed editing them, you are ready to have 
the program perform the actual optimization. Remember that OF TIM1IZE searches for a local mini- 
mum (or maximum). The absolute minimum (or maximum) may not be found. This is especially true 
for functions which have many critical points. 


When the program prompts to determine if you want a progress report, it is ready to optimize your 
function. 


Step Display Instructions 
110 Frogress report ¢''HG@3? At each iteration, the current value of the variables, the 

function value, the gradient of the function, and the 
norm of the gradient can be output. If this information 
is desired, press (Y]. This information is sent to the cur- 
rent FR IHTER I device. The default is to display the 
iterate number and the gradient norm on the current 
OISFLAY 15 device. If you get to this point and de- 
cide that you don’t want to go through with the calcula- 
tion, press (Q]. The program will return to the Optimize 
menu. 


If you don’t want a progress report, press [NJ]; the program will start iterating and you will see the 
display 


eo |Grdi: nm.nannnnE nnn 
For each iteration, the program displays the iterate number and the norm of the gradient. 


If you do want a progress report (if you press [Y]), and if you have no printer attached, you will see the 
display 


Fause on resultetY-Hi? 
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If you press [Y] in response to this prompt, the program will stop between each result until is 
pressed. If you do not want to stop, press (N] to see the progress report at the current [ELA rate. 
Either way, the program starts iterating at this point and, if you have a printer attached, the progress 
report will be printed. , 


The report will look like this if you are using an 80-column printer: 


Initial values: 


Function value =2340 


Gradient Norm =524,. 7515464946998 


No. Variable Gradient 
1 Se OOOOOOE+ 006 4, O9QQ000E +002 
2 6. OOODODDE +000 3, SQOQOOOOE +4002 


Iteration: i 


Function value =9243,85779216 


Gradient Norm =32. 2275662236 


Neo. Variable Gradient 
i 1. 73S3057E+O001 ~2,. O26680E+001 
2 1. S597326E+001 2.00571 3SE+001 


Iteration: 2 


Function value =9259. 246535081 


Gradient Norm -135. 1880682955 


No. Variable Gradient 
1 1. 680509E+001 ~1.074297E+001 


= 1.671125E+001 ~7 2 G49422E+000 
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Iterations 3 
Function value =9259. 25892476 


Gradient Norm =.161895444179 


No. Variable Gradient 
1 _1.666421E+001 4, 5262435E-902 
2 1.667023 E4001 ~1.554395E-o001 


Iterations: 4 


Function value =9259,. 25925925 


Gradient Norm =1.119234775669E-3 


No. Variable Gradient 
1 1. 46644668E+001 —-9,1GSLIS5E-004 
2 1.466667E+001 —-$. S85938E-004 


This example of a printed progress report is the progress report from “A Big Box,” the GF TIMIZE 
example in section 1, starting on page 22. 


Getting the Results 


Once the program has completed the required iterations, two situations can occur. Either the program 
will have converged or it will have reached the iteration limit without converging. 


If the program did not converge, you will see the following: 


Step Display Instructions 
1 ITERATION LIMIT Press to go to the next display. 
2 More iterations tY«Ho? Press [Y] to continue iterating (starting from the last 


iteration) with the same number of iterations as speci- 
fied in the program controls, or press [N] to return to 
the optimize menu. 


Note: You may also see a numeric computation message, such as FIT ERFR~- 
Tries > Limit. If this happens, press [f] to return to the Optimize menu. 
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If the program did converge, you will see: 


Step Display Instructions 
1 COHVERGED Press to continue. 
2 Pause on resultes¢y+H3? Press [y] if you want to use to step through 


the results, or press (N] if you want the results output at 
the current CELA rate. You will not see this prompt if 
you have a printer attached. 


As with the progress report, the final results will be sent to the current FR IHTEF 1 device. The 
report will be printed like this if you are using an 80-column printer: 


Function value =9259., 25925925 


Gradient Norm =1.11954775669E-3 


No. Variable Gradient 
1 1.666668E+001 -9, 19S1LS5E-004 
2 1.666667E+001 ~&, SE593BE-004 


These printed results are the results from “CF TIMIZE Example: A Big Box” starting on page 22. 


After the results are printed, the program will return to the Optimize menu. 


A Word on Gradient 


Since the gradient is an integral part of the optimizing process, a discussion of what the gradient is and 
how it is used in the FP Method is presented here. 


For the function 
F(X) = F(x, XQ, XB, ---y Xp) 


the gradient of F, denoted by VF, is defined by 
oF 7 Ox, 
OF / OX9 
VF(X) = | 0F/dx3 


OF /0x;, 
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Notice that the gradient of F is a vector whose length (number of elements) equals the number of 
variables. The components of VF(X) are the partial derivatives of F with respect to each coordinate. 
The negative of the gradient vector gives the direction of steepest descent (that is, the way in which X 
should be changed in order to cause the most rapid decrease in F(X)). 


You might think that the most viable approach for obtaining the next estimate for the location of a 
minimum for F is to proceed some distance from the current estimate X = (x1, X2, ..., x,) in the direc- 
tion indicated by the negative of VF(X). Indeed this technique (Steepest Descent) is in common use. 
However, this is not always a good strategy in that it can produce very slow convergence when the 
estimates get close to the desired location. The FP Method largely overcomes this difficulty by appro- 
priately modifying the gradient vector to obtain a more productive search direction. For additional 
details of the FP Method, refer to “Fletcher-Powell Method” in appendix C. 


Appendix A 


Owner’s Information 


Limited One-Year Warranty 


What We Will Do 


The HP-71 Curve Fitting Pac is warranted by Hewlett-Packard against defects in materials and 
workmanship affecting electronic and mechanical performance, but not software content, for one year 
from the date of original purchase. If you sell your unit or give it as a gift, the warranty is transferred 
to the new owner and remains in effect for the original one-year period. During the warranty period, we 
will repair or, at our option, replace at no charge a product that proves to be defective, provided you 
return the product, shipping prepaid, to a Hewlett-Packard service center. 


What Is Not Covered 


This warranty does not apply if the product has been damaged by accident or misuse or as the result of 
service or modification by other than an authorized Hewlett-Packard service center. 


No other express warranty is given. The repair or replacement of a product is your exclusive remedy. 
ANY OTHER IMPLIED WARRANTY OF MERCHANTABILITY OR FITNESS IS LIM- 
ITED TO THE ONE-YEAR DURATION OF THIS WRITTEN WARRANTY. Some states, 
provinces, or countries do not allow limitations on how long an implied warranty lasts, so the above 
limitation may not apply to you. INNO EVENT SHALL HEWLETT-PACKARD COMPANY BE 
LIABLE FOR CONSEQUENTIAL DAMAGES. Some states, provinces, or countries do not allow 
the exclusion or limitation of incidental or consequential damages, so the above limitation or exclusion 
may not apply to you. 


This warranty gives you specific legal rights, and you may also have other rights which vary from state 
to state, province to province, or country to country. 


Warranty for Consumer Transactions in the United Kingdom 


This warranty shall not apply to consumer transactions and shall not affect the statutory rights of a 
consumer. In relation to such transactions, the rights and obligations of Seller and Buyer shall be 
determined by statute. 
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Obligation to Make Changes 


Products are sold on the basis of specifications applicable at the time of manufacture. Hewlett-Packard 
shall have no obligation to modify or update products once sold. 


Warranty Information 


If you have any questions concerning this warranty, please contact an authorized Hewlett-Packard 
dealer or a Hewlett-Packard sales and service office. Should you be unable to contact them, please 
contact: 


¢ In the United States: 


Hewlett-Packard 
Personal Computer Group 
Customer Communications 

11000 Wolfe Road 

Cupertino, CA 95014 


Toll-Free Number: (800) FOR-HPPC (800 367-4772) 


e In Europe: 
Hewlett-Packard S.A. 
150, route du Nant-d’ Avril 
P.O. Box CH-1217 Meyrin 2 
Geneva 
Switzerland 
Telephone: (022) 83 81 11 


Note: Do not send units to this address for repair. 


¢ In other countries: 
Hewlett-Packard Intercontinental 
3495 Deer Creek Rd. 
Palo Alto, California 94304 
U.S.A. 
Telephone: (415) 857-1501 


Note: Do not send units to this address for repair. 
‘Service 
Hewlett-Packard maintains service centers in most major countries throughout the world. You may 


have your unit repaired at a Hewlett-Packard service center any time it needs service, whether the unit 
is under warranty or not. There is a charge for repairs after the one-year warranty period. 
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Hewlett-Packard products are normally repaired and reshipped within five (5) working days of receipt 
at any service center. This is an average time and could vary depending upon the time of year and the 
work load at the service center. The total time you are without your unit will depend largely on the 


shipping time. 


Obtaining Repair Service in the United States 


The Hewlett-Packard United States Service Center for battery-powered computational products is lo- 


cated in Corvallis, Oregon: 


P.O. Box 999 


Corvallis, Oregon 97339, U.S.A. 


Hewlett-Packard Company 


Service Department 
or 


Telephone: (503) 757-2000 


Obtaining Repair Service in Europe 


1030 N.E. Circle Blvd. 
Corvallis, Oregon 97330, U.S.A. 


Service centers are maintained at the following locations. For countries not listed, contact the dealer 


where you purchased your unit. 


AUSTRIA 

HEWLETT-PACKARD Ges.m.b.H. 
Kleinrechner-Service 
Wagramerstrasse-Lieblgasse 1 
A-1220 Wien (Vienna) 

Telephone: (0222) 23 65 11 


EASTERN EUROPE 
Refer to the address listed under Austria. 


GERMANY 
HEWLETT-PACKARD GmbH 
Kleinrechner-Service 
Vertriebszentrale 

Berner Strasse 117 
Postfach 560 140 

D-6000 Frankfurt 56 
Telephone: (611) 50041 


NORWAY 

HEWLETT-PACKARD NORGE A/S 
P.O. Box 34 

Oesterndalen 18 

N-1345 Oesteraas (Osio) 
Telephone: (2) 17 11 80 


SWITZERLAND 
HEWLETT-PACKARD (SCHWEIZ) AG 
Kieinrechner-Service 

Allmend 2 

CH-8967 Widen 

Telephone: (057) 31 21 11 


BELGIUM 

HEWLETT-PACKARD BELGIUM SA/NV 
Woluwedail 100 

B-1200 Brussels 

Telephone: (02) 762 32 00 


FINLAND 
HEWLETT-PACKARD OY 
Revontulentie 7 

SF-02100 Espoo 10 (Helsinki) 
Telephone: (90) 455 02 11 


ITALY é 
HEWLETT-PACKARD ITALIANA S.P.A. 
Casella postale 3645 (Milano) 

Via G. Di Vittorio, 9 

|-20063 Cernusco Sul Naviglio (Milan) 
Telephone: (2) 90 36 91 


SPAIN 


HEWLETT-PACKARD ESPANOLA S.A. 
Calle Jerez 3 

E-Madrid 16 

Telephone: (1) 458 2600 


UNITED KINGDOM 
HEWLETT-PACKARD Ltd 
King Street Lane 
GB-Winnersh, Wokingham 
Berkshire RG11 5AR 
Telephone: (0734) 784 774 


DENMARK 
HEWLETT-PACKARD A/S 
Datavej 52 

DK-3460 Birkerod (Copenhagen) 
Telephone: (02) 81 66 40 


FRANCE 

HEWLETT-PACKARD FRANCE 
Division Informatique Personnelle 
S.A.V. Calculateurs de Poche 
F-91947 Les Ulis Cedex 
Telephone: (6) 907 78 25 


NETHERLANDS 

HEWLETT-PACKARD NEDERLAND B.V. 
Van Heuven Goedhartlaan 121 

NL-1181 KK Amsteiveen (Amsterdam) 
P.O. Box 667 

Telephone: (020) 472021 


SWEDEN 

HEWLETT-PACKARD SVERIGE AB 
Skalholtsgatan 9, Kista 

Box 19 

S-163 93 Spanga (Stockholm) 
Telephone: (08) 750 2000 
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International Service Information 


Not all Hewlett-Packard service centers offer service for all models of HP products. However, if you 
bought your product from an authorized Hewlett-Packard dealer, you can be sure that service is avail- 
able in the country where you bought it. 


If you happen to be outside of the country where you bought your unit, you can contact the local 
Hewlett-Packard service center to see if service is available for it. If service is unavailable, please ship 
the unit to the address listed above under Obtaining Repair Service in the United States. A list of 
service centers for other countries can be obtained by writing to that address. 


All shipping, reimportation arrangements, and customs costs are your responsibility. 


Service Repair Charge 


There is a standard repair charge for out-of-warranty repairs. The repair charges include all labor and 
materials. In the United States, the full charge is subject to the customer’s local sales tax. In European 
countries, the full charge is subject to Value Added Tax (VAT) and similar taxes wherever applicable. 
All such taxes will appear as separate items on invoiced amounts. 


Computer products damaged by accident or misuse are not covered by the fixed repair charges. In these 
situations, repair charges will be individually determined based on time and materials. 


Service Warranty 


Any out-of-warranty repairs are warranted against defects in materials and workmanship for a period 
of 90 days from date of service. 


Shipping Instructions 


Should your unit require service, return it with the following items: 
¢ A completed Service Card, including a description of the problem. 


e A sales receipt or other proof of purchase date if the one-year warranty has not expired. 


The product, the Service Card, a brief description of the problem, and (if required) the proof of pur- 
chase date should be packaged in adequate protective packaging to prevent in-transit damage. Such 
damage is not covered by the one-year limited warranty; Hewlett-Packard suggests that you insure the 
shipment to the service center. The packaged unit should be shipped to the nearest Hewlett-Packard 
designated collection point or service center. Contact your dealer for assistance. (If you are not in the 
country where you originally purchased the unit, refer to International Service Information above.) 


Whether the unit is under warranty or not, it is your responsibility to pay shipping charges for delivery 
to the Hewlett-Packard service center. 
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After warranty repairs are completed, the service center returns the unit with postage prepaid. On out- 
of-warranty repairs in the United States and some other countries, the unit is returned C.O.D. (cover- 
ing shipping costs and the service charge). 


Further Information 


Circuitry and designs are proprietary to Hewlett-Packard, and service manuals are not available to 
customers. Should other problems or questions arise regarding repairs, please call your nearest Hewlett- 
Packard service center. 


When You Need Help 


Hewlett-Packard is committed to providing after-sale support to all of its customers. To this end, our 
customer support department has established phone numbers that you can call if you have questions 
about this product. 


Product Information. For information about Hewlett-Packard dealers, products, and prices, call: 


(800) FOR-HPPC 
(800 367-4772) 


Technical Assistance. For technical assistance with your product, call the number below: 
(503) 754-6666 
For either product information or technical assistance, you can also write to: 


Hewlett Packard 
Personal Computer Group 
Customer Communications 

11000 Wolfe Road 

Cupertino, CA 95014 


Appendix B 
Error and Status Messages 


The Curve Fitting Pac programs return certain messages under specific conditions. Some of these are 
merely status messages, while others occur in response to an error. An incorrectly typed or constructed 
command will produce an error message. An error in a subprogram may produce an error message and 
halt execution of the program. 


Several of the error messages are related to the amount of available memory. CF IT and OF TIMIZE 
contain tests for low memory conditions—conditions that would otherwise suspend program execution. 
If you encounter an error or warning that refers to low memory conditions, you should interrupt the 
program by pressing [ATTN], catalog the memory files, and purge unneeded files to make more memory 
available. You can then press [f][CONT] to proceed with the program. 


BASIC Error Messages 


The following is a list, in alphabetical order, of the error and status messages produced within the 
BASIC programs. Refer to page 117 for a list of the BASIC programs in this pac. 


Message and Condition 


Core 
The program has ended. The HP-71 is now ready for the next task. 


i 


ERROR: 1 <= k ¢= 8 
must be between 1 and 20. 


The number of unknowns 


ERROR: Ho 1 
The number of independent variables and data points must be positive. 


ERROR: Address 
The given array element in a GOTO command does not exist. 


ERROR: Array Too Large 
There is not enough memory to hold the new data array (or enlarge the existing data array). 


ERROR: Oata Format 
The file being read is not a properly formatted HPAF file. 
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Message and Condition 


ERROR: File Hot Found 
The file specified in a load operation or the file containing the named subprogram could not be found. 


ERROR: HPIL : 
An error was encountered when trying to print to an HP-IL device. 


FRROR: Illegal ChiS4 
The given Chi Square estimate is invalid. 


ERROR: Illegal Delta 
The given Delta is invalid. 


ERROE: Iterstian= ~ 2 
You must allow at least two iterations. 
ERROR: Limit <= 
The gradient limit must be positive. 


=, 
im! 


ERROR: Hoa Oats 
There is no data. Data may be entered with the keyboard or load option. 
ERROR: Ha Printer 
There is no printer on HP-IL. 
FREQRE: Honexletent Col 
The given column in the ADL or GELETE command does not exist. 
ERROR: Honexistent Fow - 
The given row in the DELETE command does not exist. 
ERROR: Hot Enough Mem 
There is not enough memory to edit the data, or there is not enough memory to run the program. 
ERROR: Hat HFAF File 
The data file specified for a _OAC is not in the HPAF format. 
FRROR: # Of Columns 
The file being added to the data array has a different number of columns than the array. 
ERROR: PCEWHTCHI Failed 
The calculation of the percentage goodness of fit failed. 


ERROR: SUB Hat Found 
The subprogram could not be found. 


Appendix B: Error and Status Messages 71 


Message and Condition 


ERROR: Tries < 2 
You must allow at least two line search tries. 


ERROR: ‘tou Heed 1 Row 
The data array must contain at least one row. 


ERROR: ‘You Heed 2 Cols 
There must be at least three columns in the data array—allowing for one independent variable, the 
dependent variable, and the weight. 


ERROR dd SUBFPROGRAM 
The subprogram cannot be called successfully. 


WE: Deg, Freedam < 1 
The PCENTCHI calculation is meaningless for degrees of freedom less than 1. 





Binary Error Messages 


This next listing is the error messages that can be produced when an error is detected and reported 
from within one of the FITLIE binary subprograms that you can call. In addition to displaying the 
error or warning message, the subprogram passes back a condition code in the last variable that appears 
in the calling statement. The condition code will agree in magnitude with EF FH, but will additionally 
indicate whether the exceptional event was treated as a warning or error. A negative condition code | 
indicates warning and a positive condition code indicates error. The occurrence of such a warning or 
error will not be detected by an OH ERROF statement and will not halt a running program. Con- 
sequently, it is usually best to test the condition code immediately after a call to any of the binary 
subprograms provided in this pac. CF IT and OF TIMIZE do this automatically for you. 


If CF IT or GF TIMIZE do encounter an error, the programs pause and the SUSP annunciator ap- 
pears. The program can be continued by pressing [f][CONT]. 


Note: Normal exits (including warnings and errors) will deallocate the scratch memory (system 
buffers) used by the binary subprograms making this memory available for other uses. 


However, there are four exceptional events that are treated differently than the description above. If 
the subprogram is unable to assign a value to the condition code variable (errors 3015, 3016, and 3017), 
the error message is displayed as usual, but you will not be returned to the calling environment. In- 
stead, the current file will be FI TL1IE and the environment will be that of the subprogram that de- 
tected the error. These errors are detected before any scratch memory is allocated. To recover, type 
EHOSUE from the keyboard to return to the calling environment. 
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A similar situation occurs if there is an attempt to display an error or warning and there is insufficient 
memory to do so. In this case the original error or warning message is replaced by the error message 
ERR: Insufficient Memory. Execution is halted with FITLIB as the current file and the lo- 
cal environment as that of the subprogram that attempted the error or warning message. In this last 
case, any scratch memory currently in use will not automatically be deallocated. You can recapture this 
memory by executing the keyword KILLELIFF, which explicitly deallocates all three of the system 
buffers that are utilized by this application. 


These four exceptional events are, in normal use, extremely unlikely. In fact, the OPTIMIZE and 
{FIT programs protect you from the condition code errors by internal dimensioning and value check- 
ing. The insufficient memory error is not excluded but is unlikely since the error messages are short. If 
you are short of memory, it is far more likely that you will see the error FIT ER R~~Ho Foom which 
indicates lack of memory to create required system buffers or to execute one of the many calls made by 
the binary subprograms to other subprograms. 


Binary Error Listing 


Errors and warnings detected by the binary subprograms are preceded by “F 1 T” with a “----” appearing 
where the line number is normally displayed. The following table lists the error messages that can be 
produced within the binary, subprograms. 


Condition 
Code 


Message and Condition 


FIT ERR+~Abor ted 
The key was pressed during the execution of FIT, FF, or C5. 


FIT ERR«-Pass Bu Value 
A parameter that must be passed by reference to one of the binary subprograms has 
been passed by value. 


FIT ERR«~-Wat Hot Sar 
The FP matrix must be a square matrix. 


FIT ERR«-Too Many Vars 
The number of unknowns (k) exceeds 20, or a matrix which should have length k is too 
large. 


FIT ERR~~Bad Oimension 
At least one of the arrays passed to a binary subprogram has the wrong length. 


FIT ERR«~Hoa Room 
insufficient memory to execute the binary subprogram. 
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| Condition 


Message and Condition 
Code 


FIT ERR~~Complex Mat 
An array passed to one of the binary subprograms is of COMPLEX type. 


FIT ERR~~Comples War 
An argument to one of the binary subprograms is of COMPLEX type. 


FIT ERR«~Ho Buffer 
An expected system buffer was not found after return from a call. 


FIT ERR«~~Tries = Limit 
The number of attempts within line search exceeded the supplied limit with no detect- 
able improvement. 


FIT WEH~«Tries >» Limit 
The number of attempts within line search exceeded the supplied limit but with 
improvement in the current iterate. 


FIT ERF~*Grad Delta=8 
The parameter used to approximate the gradient in GRAUF or GRADM is 0. 


FIT ERR~ me Int Tupe War 
An argument to one of the binary subprograms is of INTEGER type. 


FIT ERR«~-Int Type Mat 
A matrix argument to one of the binary subprograms is of INTEGER type. 


FIT WEH~-Gradient=a 
The matrix argument used to pass the current gradient is zero. 


FIT ERR«-CO-Int Type War 
The condition code variable is of INTEGER type. 


FIT ERR«+CC-Pass By Value 
The condition code has been passed by value. 


FIT ERR«~-CC-Complex War 
The condition code variable is of COMPLEX type. 


* These values are not assigned to the condition code variable. However, ERFH will have these values. 
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Condition 
Code 


Message and Condition 


FIT ERR«~Hal or Int 
A Hat or Imf is encountered as a computed result or as an argument where 
disallowed. 


FIT WEMW~~Grad P+dP=PF 
An attempt has been made to approximate the gradient with a value of Delta too small 
to effect a change in one of the parameters. In this case, at least one of the partials 
being approximated will erroneously be zero because of roundoff error. 


FIT ERR«~User 
A user function or model subprogram has set the condition code variable to 3020. 


FIT WREH~~User 
A user function or model subprogram has set the condition code variable to — 3020. 





Condition Code Messages 3020 and —3020 


The last two messages, 3020 and —3020, need a little more discussion to understand them completely. 


These messages provide a mechanism for handling an error or warning detected within a model or 
function subprogram you write. If the value of the condition code variable is established as 3020, the 
error is displayed and the program halts just as with other error messages generated within the binary 
subprograms. (If the value is established as —3020, the warning is displayed and the program 
continues.) 


If the condition code variable (C) is established as 11H, an error (message 3018) is generated. If the 
condition code is unchanged by the subprogram, its value upon exit remains zero and the calling pro- 
gram assumes normal processing has taken place and continues execution. Other nonzero values for the 
condition code variable assume an error if C>0, or warning if C<0, and that a message has already 
been displayed. 
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Numerical Methods 


Fletcher-Powell Method 


Both CF IT and OF TIMIZE use an algorithm commonly referred to in this manual as the Fletcher- 


Powell (FP) Method*. This eeonthn accepts a function F(X) = F(x,,x.,...,%,) with k variables and an 
initial guess P = (p,,Do,..., p,)" for the location of a local minimum, and then produces the next guess 
P* = (p,',p»',...,P;’)?. The manner in which this is accomplished is described next. 


A k by k square matrix H is first initialized as the identity matrix. The use of the matrix H and the 
manner in which it is modified after each new guess distinguishes this method from other similar 


methods. 


The negative gradient of F at the current iterate (—VF(P)) is computed. This vector gives the direction 
of steepest descent at P. A unit search direction S = (s,,59,...,8,)7 is established via 


S’ = —H(VF(P)) 
This vector is then normalized to the unit vector S via 
= S’/|S’|| 


Notice that initially H is the identity matrix, hence the direction for the first iterate is simply the 
direction of steepest descent. In theory it can be shown that the matrix H remains symmetric positive 
definite throughout the entire process and, consequently, that the function is always “decreasing” near 


P in the direction given by S.t 


*R. Fletcher and M.J.D. Powell, “A Rapidly Convergent Descent Method for Minimization,” Computer Journal, July 1963, pages 


163-168. 
+ In practice, the modifications made to H can introduce roundoff errors, thus permitting H to lose these desirable properties and 


forcing a “restart” procedure to be employed in which H is reinitialized as the identity. 
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After a useful direction S from the current iterate P has been established, the next iterate is defined as 
P’ = P +t x S for some appropriate choice of a positive scalar t. Thus P’ lies on the ray emanating 
from P in the direction S. 


The task that remains is the determination of an appropriate value for t. This task, commonly called 
line search, is not part of the FP Method. Various line search techniques are in common use. The one 
employed in this pac uses a modified cubic fit along the search ray. Details of the line search algorithm 
are given after the FP method is fully described. For now, observe that the value of ¢ should be such 
that F is minimized near P in the direction S. This observation implies that line search is equivalent to 
minimization of a function of a single variable t. The function to be minimized is 


h(t) = F(P + tS) 


After an appropriate value of t has been obtained, the iterate P’ is computed and the matrix H is 
updated as follows: (Notice that the denominators in the expressions for A and B are scalars.) 


R=P-—P; R=AP _ (kbyl) 
Q=G'-G; Q=A(VF) (k by 1) 
H=H+A-B 


The k by k matrices A & B are computed as follows: 


A = (RR*)/(R7Q) 
B = (HQQ™A) / (Q™HQ) 


Line Search 


As previously mentioned, the object of the line search is the minimization of the function h(t). Before 
launching into the explanation of the algorithm, you'll need to know some notational conventions. 
Notice again the function to be minimized: 


h(t) = F(P + tS) 


P is the current iterate and S is the search direction. The algorithm will selectively sample h at various 
values of t. Assuming selected values of t at t = to, ty, and tg, let us denote h(tp) by ho, A(ty) by h, and 
h(tz) by hy. Similarly, let us denote the slopes h’(to), h’(t;), and h’(t2) by mo, mj, and mp, respectively. 
Observe that given values to, t;, and ty, the determination of ho, hy, and hz can be obtained directly thru 
a CFILL to the subprogram that represents the function F. The values of mo, m;, and m, are computed 
via: 


m; = (VF(P + t;S))7'S; for i = 0,1,2 


What we have on entry to the line search algorithm is P, the current iterate, S, the search direction 
(from the FP algorithm), and L, the estimated functional lower bound. 
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The first step in the algorithm is to compute mg and ensure that my < 0. If this is not the case, the FP 
matrix H is reset to the identity and a new search direction S is determined. (This guarantees a new 
value of mp such that mp < 0 with equality only if the norm of the gradient is 0.) 


Then, the algorithm initializes tp = 0 (corresponding to P’ = P), and determines a value t, in such a 
manner that we expect the interval (to, t,) to be a good search interval for the function h. More specifi- 
cally, if ho < L (this corresponds to a bad guess at L), L is first reset to hg + (mp/2). Notice that mo is 
less than 0; therefore, the new value of L is less than ho. 


tz = MIN [—2(ho — L)/m ,1) 


This choice for t2 deserves some explanation. The value given by t/ = —2(hy — L)/mg is quite reason- 
able assuming that we are “close” to the minimum and L is a “good” guess at the minimum functional 
value. Near the minimum the second order terms in the Taylor expansion dominate, and h can be 
approximated by a quadratic. With these assumptions, ¢’ is the location of the minimum of the qua- 
dratic that agrees with h (both value and derivative) at 0 and has minimum value L. 


However, if the choice of L is not good, or if we are not close to the minimum, the search interval is 
restrained to maintain selective sampling near the current iterate by limiting the value of t.. The value 
of 1 for this restraint is suggested by the literature and seems adequate in practice. 


Now hy and my are evaluated (the value of h and h’ at tz). 


The four values that we now have, ho, hy, mo, mo, are sufficient to fit a cubic to h, determine the 
location of the minimum of the cubic and use this as a guess for t. 


Roughly speaking that is what the algorithm does. However, in practice it is generally more productive 
to move at least one of the endpoints in the search interval (to,t2) and try again, unless mz > 0 
(corresponding to a sign change in the derivative of h). There are various cases to consider that depend 
on the sign of mz (and the value of h» relative to hp if my < 0). These cases are illustrated by the 
following sketches and descriptions of the action of the algorithm in each case. 
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Case 1: mg > 0. This is the desirable case. 
In this case the algorithm performs the cu- 
bic fit obtaining a value t, that is an es- 
timate of the location of the minimum of h. 
In the equation 


Z= 3(ho _ ho) /(te = to) + ™Mo + Mo 


the cubic degenerates to a quadratic if 
2Z + mp + me = O. In this case: 


t, = to + molt, — to)/(2Z + 2mo) 
Otherwise 

W=\Z2 — mm, 

p= (m+ W- Z)/(2W + m, — mM) 
Also 


i= ty = p(t, = to) 


Case 2: mg < 0, hg > ho. In this case the 
algorithm changes the search interval to 
(to, (to + t2)/2). This reduces the search 
interval by 1/2. Thus, the algorithms resets 
t, to (ty +t )/2 and repeats from the 
evaluation of hy and mg. It also increments a 
counter that keeps track of the attempts to 
achieve case 1. 
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Case 3: mg = 0, hg < hg. In this case, the 
algorithm returns from the line search with 
P = P => P + toS 





Case 4: mg < 0, hy < hg. In this case the 
algorithm changes the search interval to 
(tg, tg + 2(to — tp)). This expands the search 
interval by a factor of 2. Thus the algorithm 
sets t) = t, and doubles the search interval 
size. It also, ‘as in case 2, increments the 
counter that keeps track of the attempts to 
achieve case 1 and repeats from the 
evaluation of hy and mo. 





After an attempt at a cubic fit (from case 1), you have a quite reasonable choice t, for the location of 
the minimum for h. 


If the fit was a success, t, is the location where the cubic polynomial, which agrees with h (in both 
value and derivative) at the search interval endpoints, achieves it’s minimum value. If ¢, does not lie 
within the search interval, it is reset to the interval midpoint before the process continues 
(t, = (tf + t9)/2). 


You have no guarantee at this point that t; is actually an improvement (h, is smaller) over the values 
at the endpoints of the search interval. Thus the next step is to evaluate h, and m,. 
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If hy < MIN(ho, hg), the algorithm returns from the line search with P’ = P + t,S. Notice here that 
the algorithm does not spend time attempting to improve the value of t,. In this type of application, 
determining the precise location of the minimum of h (especially for early iterations) is not nearly as 
important as minimizing the number of function evaluations per iteration. Thus, the line search al- 
gorithm attempts to produce an improved iterate and return quickly to the FP portion of the algorithm 
where a new search direction is selected. 


If, however, h, is not less than MIN(hpo, hg), the algorithm does one of three things: 


e If m, < 0, hy > ho: It contracts the search interval to (to, (to + t2)/2), increments the failure count, 
and repeats from the evaluation of hg, mp. 


e If m, < 0, hy < hg: It contracts the search interval to (t,t2) (sets ty = t;), increments the failure 
count, and retries the cubic fit. 


e If m, > 0: It contracts the search interval to (to, t;) (sets tg = t,), increments the failure count, and 
retries the cubic fit. 


Throughout the process, the failure count is getting incremented if progress is not being made. This 
value is checked against a user-supplied limit (TF: IES) each time it is incremented. If it ever exceeds 
the limit, and the line search algorithm is unable to return with a new iterate that is “better” than the 
one on entry, a fatal error results. Here “better” means that a smaller (assuming the application is 
minimization) function value results (or the same function value with a smaller value for the magnitude 
of m). 


Function Optimization 


The FP Method and line search algorithms can be applied directly to the problem of optimizing a 
function. Although the FP Method is a minimization algorithm, it can also be used to maximize a 
function. This is due to the fact that local minima (maxima) for a function F are local maxima (min- 
ima) for the function —F. 


Thus one approach to using the OF TIMIZE program to maximize a function would be to provide the 
negative of the desired function. This would require you to change the subprogram that encodes the 
function during searches for critical points. 


MAX or MHI? 


However, the user interface to the numerical algorithms is friendlier than that. You need only indicate 
that you want to find the maximum when prompted by the program. The interface will then set or clear 
the appropriate user flag that is examined at the time that F is evaluated. If appropriate, the value of F 
(and it’s gradient) will be negated based on the value in this control flag. This control flag is user flag 
number 61. 
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Gradient 


You can optionally indicate that you want the gradient of your function (or model) approximated. This 

_is also implemented in the user interface through the use of a user flag that is examined at the time the 
function (or model) is evaluated. This flag is user flag number 62. The user interface responds to your 
preference by setting or clearing this flag. 


However, you should be aware that the use of this friendly feature to approximate the gradient can 
make the program run significantly slower. Information on gradient approximation follows. 


Gradient Approximation 


If you are unable to compute the gradient in the subprogram that you use to specify your function (or 
model, if your application is curve fitting), you can have the gradient approximated for you. This can be 
very convenient, but has speed and accuracy considerations that you should be aware of. 


Speed 
Let’s take an example using F IT, assuming 20 data points and the model function given by 
F = FL#5THCRP2EXS + PSEERPC-HS 


If the gradient is to be approximated, at least k + 1 calls (k = number of parameters) to the model 
function will be necessary to obtain the value for F and it’s approximated gradient. In the above exam- 
ple, k = 3 and four calls are necessary. One call establishes the function value F. Three additional calls 
are necessary to evaluate F near P() in the three coordinate directions. For example, the approxima- 
tion of the first coordinate in the gradient requires 


F((P, + 6 Po, P3),X) — F 


€ 


aF/aP, = 


for some appropriately small non-zero value ¢«. Consequently, four calls to the model are required where 
only one would be required if the gradient were computed in closed form within the subprogram. 


If this example is carried a bit further, the impact on speed can be seen more clearly. The function to be 
minimized is the Chi Square function that is associated with the model. The Chi Square function needs 
to call the model subprogram 20 times, each time obtaining both a model value and the gradient of the 
model to produce the Chi Square function value and it’s gradient. If the gradient of F must be approxi- 
mated, at least eighty (4 x 20) calls to the model subprogram are required to achieve each value of Chi 
Square and it’s gradient. Examination of the line search algorithm shows that generally at least two 
functional evaluations will take place (and often more) for each iteration. 


At the minimum then, for 20 data points, 160 calls will be made to the model function per iteration. 
This compares to 40 calls if the model gradient is supplied within the subprogram. 
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Most of the computation time will be spent within your BASIC subprogram. Consequently, anything 
you do within this subprogram to effect speed improvements (including the evaluation of the gradient 
in closed form) can have a significant impact on the total speed of the program. 


Accuracy 


One of the drawbacks to the approximation of the gradient is that the difference quotients can be a 
source of significant error. To illustrate, when the values in the numerator of the following equation are 
sufficiently close, 


F((P, + €, P., P3), X) are F 


€ 


aF/aP, = 


their difference can produce 0 when the actual value can be quite large. The value entered in CF IT and 
CP TIMIZE as the constant or percentage is referred to here as Delta. If flag 63 is clear (Delta = 
constant), the value of « is Delta. If flag 63 is set (Delta = percent), « is the specified percentage of the 
parameter P as shown below: 


| Px (elta/100) if P#0 
€= | Delta/100 if P=0 


The main sources of error in the previous equation are the calculation of P, + « and the difference that 
appears in the numerator. Both of these sources of error can require special handling to avoid 
inaccuracy. 


To illustrate these errors, consider the box example on page 22. The first line of the following table 
shows the result if VV(W,H) is computed at W = H = 16.66664 within the subprogram by a direct call 
to the given box subprogram. The value in line 1 is the exact value of the gradient vector and is useful 
for purposes of comparison with the results presented in the lines of the table representing gradient 
approximation (lines 2, 3, and 4). These last three lines were obtained by calls to the subprogram 
GRACIF with flag 63 (relative approximation) set and with the indicated values for Delta. 










VV(W,H) = (0V/aW,dV/dH)* 
avjaw 8V/dH 





Delta (%) 









(actual value) 2.66666240000 E—3 2.66666240000 E—3 
0.001 —3.06000489601 E—3 —3.06000489601 E—3 
0.0001 1.80000288000 E-—3 1.80000288000 E—3 
0.00001 0 0 
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This table illustrates some of the pitfalls that can beset the unwary user of gradient approximation. 
The previous equation shows the first coordinate (9V/dW) of VV(W,H) is computed as 


ov = = V(W + «,H) — ViW,#) 

aw € 
where « = W x (D/100). In this case the basic problem can be traced to the fact that the point (W, H) 
is very close to the local maxima at (W,H) = (50/3,50/3). 


V 








fess - Line 
A 
eet 






The thing to realize here is that the secant line joining the points (W, H, V(W,H)) to (W+., 
H, V(W + «, H)) is being used to approximate the partial of V with respect to W. As in the two- 
dimensional case, where a secant is often used to approximate the derivative of a function, the choice 
of « can become critical near local minima or maxima. 


In fact, if « is too large, the secant line can become a poor approximation and yield unsatisfactory 
‘results (as in the second and third lines of the table). If « is too small, the function may be unable to 
distinguish between (W,H) and (W + «,H) due to roundoff error in the functional evaluation. In this 
case, the difference quotient becomes zero (as in the fourth line of the table). 


If possible, compute the gradient within your subprogram—especially if you need extremely accurate 
results. 


84 Appendix C: Numerical Methods 


Application to oF iT 


The FP Method and line search algorithms described previously can be applied directly to a curve fit 
problem. This is true because there is a binary subprogram named (2 in the file FITLIE that 
computes Chi Square (and it’s gradient) of the user’s model and data set. This subprogram, as well as 
others in the file FI TL1&, can be called from the keyboard and called from within user-written pro- 
grams. For information on 0 and other user-accessible routines, refer to appendix D. 


Minimizing Chi Square 


CFIT attempts to fit your data by minimizing the Chi Square function associated with your data set 
and the specified model. 


“The Chi Square function associated with the model F=F(X( ),P()) at the current parameter iterate 
P() is defined by 


x(P()) = = [(F,— Y)/W,P 


(1) Chi Square Function 


Where F; denotes F = F(X(),P()) with X() the ith row in the Data Set. Y; is the dependent variable 
and W; is the weight. 


dx7(P() i AB = Y). OR; 
es ex SSeS ck 
éP; 2 Ww? aP; 
(2) Gradient Chi Square Function 


With equal weights, equation (1) corresponds to the function that is minimized in the usual least 
squares method. Equation (2) provides the gradient of the Chi Square function in terms of the gradient 
of the model. Observe that there is no requirement that the model be linear. Indeed the model function 
may be quite general. 


Difficult Cases 


In addition to the difficulties associated with gradient approximation (refer to page 81 for information), 
there are areas where you may experience difficulty in obtaining the desired results. In some cases the 
difficulty can be avoided or its impact minimized. Three of these situations are described below. The 
information is not designed to provide a general solution to all the problems you might encounter, but 
to give you some ideas as to how to go about solving problems. 
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The Inability to Meet Convergence Criteria 


The nature of the function being optimized (or the nature of Chi Square in [F IT) and the limits of 
machine precision can combine to keep the gradient norm from becoming sufficiently small to meet the 
convergence criteria used by OF TIMIZE and CFIT. 


It is not unusual for the current iterate to be very close to the desired result, yet still have a reasonably 
large value for the norm of the gradient. This makes the selection of the limiting value used by CF IT 
and UF TIMIZE to detect convergence somewhat arbitrary. For this reason, you are given the ability in 
both CF IT and GFTIMIZE to change the value for the gradient limit by editing the program controls. 
A good strategy is to start with the gradient limit large, and, after convergence, reduce the limit and 
continue in an attempt to get closer to the desired result. 


The error message 
FIT ERR~-TEIES > LIMIT 


often indicates that, because of machine precision and because of the nature of the function, the cur- 
rent iterate cannot be improved. 


Sampling Outside the Intended Domain 


An attempt to minimize the function F(x, y) = (x ~— 3)? + V(x — y) without special precautions will 

almost certainly result in sampling the subprogram for F at some point for which x < y. This, in turn, 

will result in the math error S/F ¢ rea %. The solution for this type of problem depends on the nature 

of the function. For this example, probably the most simple solution involves the realization that you 
ean replace the \/(x — y) with (x — y)? without altering the solution. 


Constrained Optimization 


Some functions are subject to equality and inequality constraints. For example, a function to minimize 
such as F(x, y) = (x — 3)? + (y — 2)? can be subject to the constraint G(s, y) =x+y—4=0 or, pos- 
sibly, x + y—4<0. 


In simple cases like this the equality constraints can be used to solve for one variable in terms of the 
others. Substitution into the object function F will then eliminate one or more variables, and then the 
resulting function can be minimized. 


Inequality constraints, such as G(x,y) < 0, are often handled by the use of a penalty function. One or 
more terms can be added to F to penalize samples taken from the wrong or undesirable region. For 
example: 


A(x,y) = F(x,y) + R/G(x,y) 
= (x — 87 + (y— 2)? + R/(x + y — 4) 
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A minimum for H when R = 1 can be found using OPTIMIZE. By successively reducing R in small 
steps and by using the value obtained from the previous solution as an initial guess, you will obtain 
solutions which converge to the answer (x, y) = (2.5,1.5) where F(x,y) = 0.5. This type of procedure 
could even be “automated” by storing the updated value of R in a file that would be read by your 
subprogram for H. 


Appendix D 


User-Accessible Routines 


This appendix describes the numerical subprograms that are available in the file FITLIE, their syn- 
tax, their calling relationships, and their memory requirements. Also, there is information describing 
the BASIC subprogram that produces the value of the Chi Square distribution function as well as 
information on two BASIC keywords, KE‘‘HAIT# and KILLELFF, which reside in FITLEX, 


Subprogram Description and Calling Syntax 


_ What is described here is the required syntax in a CALL statement to the user-accessible subprograms. 
Of course, the actual variable names that you choose are immaterial. What is important is that the 
number, type and location of the arguments agree with the specifications that follow. In the case of the 
array parameters, the dimensions must be consistent with the problem. For example, the length (the 
number of elements) of the array B() that appears in the call to £3 must equal the number of un- 
known parameters. 


While none of the binary subprograms are base option dependent, CF IT and OF TIMIZE require base 
option 1. 


The FF Subprogram 


Functional Description. Performs one iteration of the FP method and the line search (described in 
appendix C) upon the function specified by the subprogram Ali # located in the file A= ¢#. 


CALL FROAL$, Ae#, Bo? ,009,0,6,F¢,3,6,H,1,23 IH FITLIB 


Inputs: 
Fiil#- Name of the subprogram encoding the function. 
Ha - Name of the file that contains the subprogram. 
Bes - Current iterate. 
(<3 - Gradient at the current iterate. 
C - Delta for gradient approximation (if flag 62 set). 
E - Function value at the current iterate. 
Fe, > - FP matrix. 
G - Line search iteration limit. 
H - Estimate at functional bound. 
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Control Flags: 


Set | MAX Approx dP = % 
Clear | MIN | No Approx | dP = Constant 








Outputs: 

Ec 2 - New iterate. 

i:< 3 - Gradient at the new iterate. 
E - Function value at the new iterate. 

Fe, 3 - Updated FP matrix. 

H - Updated bound estimate if original “bad.” 
I - Norm of the gradient. 
= - The condition code. 


Comments: If the function has k vari- Array Length 
ables, the arrays must have dimensions BC) k 
that produce the lengths shown at C() k 
right. F(,) kbyk 


The FP subprogram samples the user’s subprogram that encodes the function to be optimized by calls 
of the form: 
CALL ALFCBCa, Ceo, £, 23 IH Aet 


The subprogram FP uses the results of such calls as described under “Fletcher-Powell Method,” in 
appendix C. 


The GEACF Subprogram 


Functional Description: Produces an approximate gradient vector © « + associated with the function 
specified by the subprogram Fii # that appears in the file Hz#. 


CALL GRAOFCA1$,AZ$,6¢3,0¢9,0,E,23 IN FITLIB 


Inputs: 
fii¢- Name of the subprogram encoding the function. 
Fiz - Name of the file that contains the subprogram. 
E> - Current iterate for the unknown parameters. 
Ci - Delta for gradient approximation. 
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Control Flags: 


Set not used | not used dP = % 
Clear | not used | not used | dP = Constant 
- Outputs: 


{¢ + - Approximated gradient at E«:. 
— - Function value at B« :. 
= - The condition code. 





Comments: An error is returned if Array Length 
D=0. | B() k 
C() k 


The jth coordinate, Cj is evaluated by the formula: 
feel F(b,,b,,...,0; + €,---D,) — F(d, bo ,.-. Bj ,-+- By) 


J 
€ 


F is the function whose gradient is being approximated. « = D if flag 63 is clear. If flag 63 is set, 
e = 6(D/100) or D/100 if b; = 0. 


The GRAOF subprogram is called by FP in the case where the gradient is to be approximated. The 
‘function subprogram must have the syntax that corresponds to the following call: 


CALL ALS$¢Bes,Co9,E,23 IN Ags 


The function subprogram A1i# will be called k + 1 times to accomplish the stated objective. 


The FIT Subprogram 


Functional Description: Performs one iteration of the FP Method and the line search upon the Chi 
Square function associated with the specified model and data set. 


CALL FITCALF, Aft, Jo, aos, Boo, Ces, 0,€,Ft,3,6.,H,1,2% IH FITLIEB 
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Inputs: 
fi1# - Name of the subprogram encoding the model. 
fi2£ - Name of the file containing the model subprogram. 
«, 2 - Data array. 
#3 - Seratch array. 
«3 - Current iterate for the unknown parameters. 
is - Gradient at the current iterate. 
(- Delta for gradient approximation (if FLAG = set). 
E - Chi Square value at the current iterate. 
Fe, 3 - FP matrix. 
i - Line search iteration limit. 
H - Estimate at functional bound. 


2 Control Flags: 


MAX Approx dP = % 
MIN | No Approx } dP = Constant 






Outputs: 

E¢ > - New iterate. 

is - Gradient at the new iterate. 
E - Chi Square function value at the new iterate. 

’ Fé, » - Updated FP matrix. 

H - Updated bound estimate if original “bad.” 
1 - Norm of the gradient (usual norm). 
2 - The condition code. 


Comments: FLAG 461i should be Array Length 

clear (set equal to 0) so that the Chi J(,) m by (n + 2) 

Square function is minimized. If the X() n+2 

model has k parameters, n independent Bi) ie 

variables, and there are m data points, C() k 

the arrays must have the lengths F(,) k by k 

shown at right. 
The FIT subprogram calls the ¢( subprogram which in turn makes calls to the model subprogram 
in order to achieve it’s task—performing one iteration of the FP algorithm applied to the Chi Square 
function associated with the user’s model. The calls made by FIT to [5 have the form: 

CALL. CSech iS; Hee elt 3. Chy Bay OC 0 Exe) VI Fitts 


The subprogram FIT uses the results of such calls as described in “Fletcher-Powell Method,” in 
appendix C. 
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The (3G Subprogram 
Functional Description: Evaluates the Chi Square function and its gradient associated with the 
‘specified model and data set at B« 2. 


PCALL CSQCA1#, ARS. 0,9,809,803,009,0,£,23 IN FITLIB 


Inputs: 
| Aiil#- Name of the model subprogram. 
Ai - Name of the file that contains the model subprogram. 
J¢, 3 - Data array. 
He - Scratch array. 
B«¢ > - Parameter array. 
[1 - Delta for gradient approximation (if FLAG 2 set). 


Control Flags: 


pele eee 





Set | not used Approx dP= % 
Clear | not used | No Approx | dP = Constant 


Outputs: 
3 - The gradient of Chi Square at Eu >. 
E - The value of Chi Square at E« ?. 
=< - The condition code. 


Comments: If the model has k param- Array Length 
eters, nm independent variables, and B() k 
there are m data points, the arrays X() n+2 
must have the lengths shown at right. J(,) m by (n + 2) 
C() k 
The #« ° variable is used by = to successively load in rows of the data array and execute calls to 


the user’s model subprogram via: 
CALL AL#¢BO3,809,009,6,2) IM Aes 


The results of each such call are used to update sums kept for the values of Chi Square and it’s gradi- 
ent. 
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The GEACH Subprogram 


Functional Description: Produces an approximate gradient vector ~ « » for the model function Ai # 
in file RZ#. 

CALL GREAOMCAL#$, Aft. #oo, Bea, oes, 0,6,22 IH FITLIE 

Inputs: 


fil #- Name of the subprogram encoding the model. 
A2# - Name of the file that contains the model subprogram. 
“<¢ 3 - Scratch Variable used to pass row data to model. 
E&« > - Current Iterate for the unknown parameters. 
t'- Delta for gradient approximation. 


Control Flags: 


not used | not used dP= % 
not used | not used} dP = Constant 
Outputs: 


is - Approximated gradient at E« ?. 
E - Model value at Et 2. 
= - The condition code. 








Comments: An error is returned if 


Array Length 
D=0. X() n+2 
B() k 
C() k 


The jth coordinate, ¢,, is evaluated by the formula: 


F (By, Bg, .0 9D; + 62 Dgs X45 Xa 00+ 9% ) 


,X,) — F(b,,bo,...,0j,-..Dy3 Kip Keay ass Xp) 
Cc: = 
j 


€ 


F is the model whose gradient is being approximated. « = D if flag 63 is clear. If flag 63 is set, 
« = b(D/100) or D/100 if 6; = 0. 


The GEFIOM subprogram is called by (5 in the case where the gradient of Chi Square is to be 
approximated. The model subprogram must have syntax corresponding to the CALL: 


CALL ALSCBbes, #03, 003,6,2> IN AeF 
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The model subprogram Aii# will be called k + 1 times by GFALIM to approximate the gradient of the 
model at a particular fixed row of the data array. The gradient of Chi Square is approximated by 
repeating this process m times (once for each data point or row in the array) and summing the results. 
A call to £38 with the gradient of the model to be approximated will result in m x (k + 1) calls to 
the model subprogram. 


The FOL‘ Subprogram 


Functional Description: Computes E, the polynomial value at x, and = «>, the gradient with respect 
to the coefficients. The degree (up to 19) is determined by array sizes, and the evaluation proceeds by 


Horner’s method. 
CALL POLY Bes, 8e5,005,6E,23 IH FITLIB 
Where E and © «> are returned as: 
| E = b, + box + bax? +... + b,x") 
C() = (1, x, x2, ..., x® 7-4) 


Inputs: 
B« > - Coefficients (b,,5,,...,0,). 
Hed - (x,y,w). 
Outputs: 
i ¢ 3 - Gradient with respect to coefficients. 
E - Polynomial value. 
-2 - The condition code. 
Comments: Array Length 
X() >1 
B() k 
C() k 


The L Iti Subprogram 

Functional Description: Contains linear models of up to order 19. 
CALL LIW¢ Bes. eeo,009,E,23 IN FITLIE 

Where E and (¢® are returned as: 


E = b,x, + boXq, + «+ Die Xk — 1) + b, 
C() = (x, Xo» eeey X(k — 1) 1) 
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Inputs: 
*%« 2 - Independent variables (k - 1 of them), dependent variable (Y), and weight (W). 
Ei > - Linear coefficients (k of them). 
Outputs: 
(3 - gradient. 
E - Linear value E. 
2 - The condition code. 
Comments: The array **% is as- Array Length 
sumed to contain Y and W even X() k4F1 
though these values are not used by B() k 
LIW. C() k 


The FCEHTCHI Subprogram 

Functional Description: Evaluates the Chi Square distribution function, P(x, v). 
CALL PCEMTCHICH WP, 23 

Inputs: 


# - Value of Chi Square. 
\! - Degrees of freedom. (\\ must be a positive integer.) 


_ Outputs: 


F - Percentage of Chi Square distribution less than the given Chi Square value (for the given 
degrees of freedom). The value returned is rounded to the third decimal place. 
= - Error code (0 = okay, nonzero = error). 
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~ Comments: The Chi Square density function is really a family of curves—one for each positive degree 
of freedom. The ‘cumulative distribution function is the area under the curve from zero to a given value 
of Chi Square. 


If x < 0, then P=0O. 
If v = 40, then the value of P is computed by using a quick approximation. 


Let 


i = Labs(x)/v)'? — (1=2/(90)) 
V (2/(9v)) 


Let ND be the normal distribution function. Its value at x can be approximated as follows: 


p= 5(1 + dx + dox? + dax? + dyxt + dgx° + dgx®)~ 1% 


Where 
d, = .0498673470 
dy = .0211410061 
d3; = .0032776263 
d, = .0000380036 
ds = .0000488906 
dg = .0000053830 
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If x > 0, ND(x) = p. If x < 0, ND(x) = 1 — p. 
Finally, P(x,v) can be defined as: 
P(x, v) = ND(x’) 


This approximation is derived from the assumption that the Chi Square distribution for large degrees 
of freedom can be modeled by the normal distribution function with appropriate mapping of the 
domain.* 


If v < 40 and x > 80, then P(x,v) = 1 which is accurate to the required three digits. 


If v < 40 and x < 80, the value of P is computed using the following finite series:t 


If v is odd: 


(v-1)/2 ( x) 2D 
P(x,v) = 2xND(Vx) — 1+ V(2/r) x exp(-x/2)x > a2 EEE Ss 


MM = 1%* 3x5... x (2-1) 


Where ND(x) equals the left tail normal distribution computed according to the formula used 
previously. 


If v is even: ‘sag yp 
P(x,v) = 1 — exp(-x/2) o| (or > (Vx)? | 


| aol 2x 4x6... x 2i | 


* For information refer to the Handbook of Mathematical Functions by Abramowitz and Stegun, National Bureau of Standards, 
1968, equations 26.2.19 and 26.4.14. 


+ Abramowitz and Stegun, Handbook of Mathematical Functions, National Bureau of Standards, 1968, equations 26.4.4 and 26.4.5. 
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Calling Relationships 


_ The subprograms in the file FITLIE (FF, FIT, C50, GRADF, GRADM, POLY, and LIHW) are called 
from within CFIT and GF TIMIZE and also call each other. The following diagrams show the 

relationships between these subprograms. Arrows pointing down in the diagrams represent calls (mul- 
tiple calls where indicated) to the lower level subprograms. 


_Subprograms Called by FF 


Optimization 






FLAG 62 SET 


FLAG 62 CLEAR 


User 
Subprogram 


Flags 61, 62, and 63 are inputs acting as control flags, and k is the number of function variables. 






(k+1 times) 


The user subprogram computes the function value and the gradient of the target function. If the gradi- 
ent is to be approximated, a call to GRALIF replaces a call to the user subprogram. The syntax for a 
call to the user subprogram is as follows: 


CALL ALS¢Bes,0Ceo,6&,2,2 IW Ast 


The variables in the call are described in the discussion of FF’, starting on page 87. 
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Subprograms Called by FIT 


Curve Fitting 










FLAG 62 SET 
(m times) 


GRAOM 


(k + 1 times) 


FLAG 62 CLEAR 





(m times) 
Binary Models 
Model — 
Subprogram LIH 


Flags 61, 62, and 63 are inputs acting as control flags, k is the number of unknown model parameters, 
and m is the number of data points. 


The model subprogram (either user-written or provided in ROM) computes the model value and the 
gradient of the model with respect to the unknown parameters for a given data point. F I T functions in 
much the same way as FF, except that the target subprogram for minimization is not the user sub- 
program, but is C'5@. If the model gradient is to be approximated, calls to GRAOM replace the calls to 
the model. The syntax for any model subprogram is identical to that for POL’ and LIN. 


Memory Requirements 


There are seven binary subprograms in FI TLIE. These subprograms are called by the programs 
FIT and OPTIMIZE, but you can also call them from the keyboard or within your own program. 
What follows is information on the buffer and calling overhead associated with each routine and 
information on memory requirements for variable storage. 


Buffer and Calling Overhead 


Five of the seven binary subprograms create workspace buffers in main memory at execution time. The 
buffers are deallocated on exit, releasing this memory back to your system. Overhead memory is re- 
quired by the operating system for each subprogram CALL. The buffer size (including header) and 
calling overhead are provided in the following table. The buffer size is a function of the number of 
unknowns (k), and the overhead is a function of the sum of the lengths (j) of the two string arguments 
in the call. 
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Buffer and Calling Overhead 


Buffer Size | Call Overhead 


318 + 189k | 301+ 3 (619 + 189k + 2/)/2 
FIT 318 + 189k | 339 + 3 (657 + 189k + 2/)/2 
csa bCHISQ | 108 + 21k | 263 + 2 (371 + 21k + 2/2 
GRADF bGRAD | 95+ 21k | 225 + 2 (320 + 21k + 2/2 


GRAOM bGRAD | 54+ 21k | 2444 2 (298 + 21k + 2)/2 


FOL’, LIH | None 0 179 179/2 
and other 
{FIT models 


OFTIMIZE 160/2 
subprograms 


* One nibble is equal to 1% byte of memory. 





Notice that FF and FIT share the same buffer, as do GF-ADF and GFAUM. Also, more than one buffer 
can be active at any given time. After a call to FIT, if the model gradient is to be approximated, all 
three buffers will be simultaneously active. 


Variable Memory 


In addition to the overhead associated with setting up a local environment within the subprogram, 
memory is required for the variables that appear as arguments in the calling statement. The amount of 
memory required for a numeric variable depends on the number of elements (7) in the variable (n = 1 
for a simple numeric variable). The amount of memory required for a string variable depends on the 
length (J) of the string. The following table shows the memory requirements for variable storage. 


Variable Storage 


Type of Variable | Memory Required (Nibbles) 


19 + 16(n) 
19 + Qn) 
23 + 2(/) 
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As an example, consider a two-dimensional Real array that is 20 by 5 (has 100 elements). This array 
would require © 


19 + 16(100) = 1619 nibbles 


which is a little less than 810 bytes of memory. 


Keywords 


There are two keywords in FITLE#: KEVYHAIT# and KILLBUFF. To run CFIT and OPTIMIZE 
from memory (without the module plugged in), you must have several other files including FITLEX 
and FI TLIE in memory as well. 


KEVYHMAIT$ 


When the KE HAIT# function is executed, the HP-71 goes into a low power consumption state until 
a key is pressed and then returns the key name. This is similar to KE‘'#. 


EILLBUFF 


If a binary subprogram does not normally terminate when it encounters a low memory condition, the 
scratch memory allocated to it may not be deallocated. In this rare circumstance, the scratch memory 
can be reclaimed by executing KILLBUFF. 
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Library Subprograms 


In order to provide easy specification of the more commonly used fit models to CF IT, a collection of 
subprograms that correspond to these models has been placed in built-in library files in this pac. The 
functions in the library files cannot be used in the CF TIMIZE program because OF TIMIZE and 
CFIT require different syntax (the number of parameters in the CALL to the subprograms differ). 


If your model corresponds to one of the subprograms in one of the library files, you need not write a 
BASIC subprogram for it. When the program prompts you, you need only provide the name of the 
model subprogram that already exists. When you use the built-in models, remember that all models 
provided in this pac compute the model gradient as well as the model value. 


Note: The built-in models with trigonometric functions put your HP-71 into radians mode. 


The file FITLIE contains two subprograms, FiIL' and LIN, that provide all the polynomial and 
linear models of one variable that (F IT can handle (a maximum of 20 model parameters). Appendix 
D, “User-Accessible Routines,” contains information on F(iL‘’ and LIH, starting on page 93. Also, 
there are 46 additional models in the file 400ELS. These are described in the following table. In the 
model functions, a represents the first parameter (P(1)) in the equation, b the second (P(2)), and so on. 


When you are using these models, don’t forget that (FI T makes its fit by finding the local minima of 
the Chi Square function. The “absolute best fit” may not be found. Instead, (FIT may converge to a 
“local best fit.” This situation is most likely when you’re using models containing periodic functions. 
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_ 
ft 


ORPARAE 





2 1 QELIWE 

3 2 HYPER 

4 3 SHYFER 

5 2 POWER 

6 3° OFOHER 

7 4 FOWERZ 

8 6 FOWER 3 

9 2 MPOWER 
10 3 OMFPOWE Fr 
11 2 RLINE 

12 3 LOGISTIC 
13 2 ROOT 

14 2 SUPERGEO 
15 2 RTGEQ 

16 3 LIWHYVFER 
17 2 RHYFER 
18 2 ERP 

19 2 AE SF 

20 3 DOE RP 





No. of 






Set of Models Supplied in Pac 





Description 





Parabola through origin. 


Line through origin. 


Hyperbola. 


Second-order hyperbola. 


Power. 

Offset power. 

2 term power. 

3 term power. 
Modified power. 
Offset modified power. 
Reciprocal line. 
Logistic. 

Root. 

Super geometric. 

Root geometric. 
Linear hyperbolic. 
Reciprocal hyperbola. 
Exponential. 
Asymptotic exponential. 


Double exponential. 


Model Function 


y = ax + bx? 
y = ax 
y=a+tb/x 


y =a + b/x + ¢/x? 


y =ax? 
y =a-+t bx® 
y = ax? + cx? 


y = ax? + cx? + ex! 


y = ab” 
y=a-+ be" 
y = 1/(a + bx) 


y = a/(1 + be*) 


y= ab" 
y = ax) 
y = axl" 


y=axt+b+e/x 
y = x/(a + bx) 

y = a exp(bx) 

y = a(1 — exp(bx)) 


y = a (exp(bx) — exp(cx)) 
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Set of Models Supplied in Pac 


No. of 


ETEXF Root exponential. y = a exp(b/x) 









LINESF 





Linear exponential. y = ax exp(bx) 










Los Logarithmic. y =a + bin(cx) 





24 FLOW Reciprocal logarithmic. y = 1/(a + b In(cx)) 





25 HOERL Hoerl function. y = ab*x° 





26 FTHOERL Root Hoerl function. y = abye 














HOR PMO 


Normal distribution. y = aexp((x — b)?/c) 








LOGHORME 





Log-normal distribution. y = a exp((In(x)—b)2/c) 





29 BETAD Beta distribution. y = ax®(1—x)° 





30 CANMAD Gamma distribution. y = a(x/b)° exp(x/b) 







31 CAUCHYO — Cauchy distribution. y = 1f(a(x + b)?+c) 








SIAL Sinusoid. 





-y =asin(b + cx) 














OSTHU 





Offset sinusoid. y =asin(b + cx) +d 










LOST Htl 





Cosinusoid. y = acos(b + cx) 






OcoOs TH 





Offset cosinusoid. y =acos(b + cx) +d 
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Set of Models Supplied in Pac 


No. of - 
BES Model Name Model Function 


36 3 SIHF2 2 term sine Fourier. y=b+csin(ax) 






Description 






37 SIHFS 3 term sine Fourier. y = b+ sin(ax) + d sin(2ax) 


38 SIHF 4 4 term sine Fourier. y = b+ csin(ax) + d sin(2ax) + e sin(3ax) 









40 COSr a 3 term cosine Fourier. y = b+ ccos(ax) + d cos(2ax) 











4 
5 

39 3 COSFe 2 term cosine Fourier. y = b+ ccos(ax) 
4 
5 


41 COS 4 4 term cosine Fourier. y = b + c cos(ax) + d cos(2ax) 


+ e cos(3ax) 








42 4 FOURIER 2 term Fourier. y = b/2 + c cos(ax) + d sin(ax) 





43 6 FOURIER 3 term Fourier. y = b/2 + c cos(ax) + d sin(ax) 
+ e cos(2ax) + f sin(2ax) 


44 8 FOURIER4 4 term Fourier. y = b/2 + c cos(ax) + d sin(ax) 
+ e cos(2ax) + f sin(2ax) 
+ g cos(3ax) + h sin(3ax) 






Hyperbolic cosine. y =a-+cosh(b + cx) 


Hyperbolic cosecant. 





y =a + sech(b + cx) 


Suppose you suspect that the data you want to fit a curve to tends to a parabola through the origin. 
Rather than immediately setting out to create a model function for a parabola through the origin, you 
should look through the table above. The table shows that the model function has been written for you 
and is called GE FARAE in the file HOCELS. So, you don’t have to write any BASIC routines. Instead, 
when the curve fit program asks for the name of the model and the file, you type ORPARAE and 
MOOELS, respectively. 
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Applications File Format (HPAF) 


The HP-71 Curve Fitting Pac stores data in files according to a prescribed format. This format, the 
applications file format (HPAF), is intended to allow exchange of data between various programs. The 
format provides room for information that describes the structure of the data so that various programs 
may make use of and exchange the data. 


HPAF files are of type DATA, and may reside in either the HP-71 memory or a mass storage device. 


The HPAF files are composed of three major sections—the header, the data records, and an optional 
descriptor block. An example of such a file is described in the following table. 


[Res] tomes [ein 
0 


“HPAFNNS’” Type string: two numbers, one string. 
4 There are four records of data. 

12 The descriptor block starts at 12. 
77,9.3,“RED” First data record. 

78,9.4,“BLUE” Second data record. 
81.5,10.3,“GREEN” Third data record. 
82.9,10.4,“GREEN” Last data record. 

Empty data records. 

Empty data records. 

Descriptor block. 










































“TEMP”,1,“KELVIN’ 
“COLNAMS’,3,“TEMP” 
“VISCOSITY”,“COLOR’ 
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Header Information 


The header must contain the following items: 


Record 0 contains a type string. The first four characters indicate the file is an HPAF 
file. The remaining characters describe the number of data items in each record and 
their type. For example, in “HPAFNNS” the characters “NNS” indicate that there are 
three items in each record (the first two are numbers and the third a string). 


Record 1 contains the number of data records that contain information. This number 
can be less than the total number of available records (allowing room for additional 
records to be added later followed by the optional descriptor block). 


2 contains the record number of the optional descriptor block. If no descriptor block is 
present, this number should be zero. 





Data Records 


The data records start at record 3 and must end before the descriptor block. Note that all data items 
for each record must fit within each logical record so that any record can be accessed randomly. To 
compute the optimal logical record length for the file, remember that each number written in the 
record occupies 8 bytes, and each string occupies 3 bytes plus the number of bytes in the string. For 
example, if each record is going to hold two numbers and a ten character string, the record length must 
be at least 2 x 8 + 3 + 10, or 29 bytes. For more information about creating data files, refer to “Data 
Files,” in section 14 of the HP-71 Owner’s Manual. 


Descriptor Block 


The descriptor block is optional. If present, the descriptor block must come after the data records, and 
record 2 must contain the record number of the first item in the block. Information in the descriptor 
block consists of tags, which identify the type of information that follows; followed by the number of 
items associated with the tag; followed by the items themselves. 


TAG,number of items,item one,item two... 


The information in the descriptor block can be written serially, or, if the logical record size is suf- 
ficiently large, written one tag to a record. In either case, the descriptor block must be able to be read 
serially. 
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For example, to describe the names of the columns and the fact that the units are in degrees Kelvin, the 
descriptor block for the above file might look like this: 


67 “COLNAMS’,3,“TEMP”,“VISCOSITY”,“DENSITY”, 
“XYOFFSET’,2,4, — 3, 
“DEGREES’,1,“KELVIN” 





















Column names: Temp, viscosity, dénsity. 
X-Y offset: (4, — 3). 
information on units: Degrees Kelvin. 





The Curve Fitting Files 


The HP-71 Curve Fitting Pac can read any HPAF file. String-type data items will be ignored auto- 
matically. For instance, if a file has a string NSSNSNN, columns 1, 4, 6, and 7 will be considered the 
first, second, third, and fourth columns in the data array for the CF IT program. The first and second 
columns would be interpreted as the independent variables, the third column as the dependent variable, 
and the fourth column as the weight. 


When writing to a data file, CF IT generates no descriptor block, so the record number in record 2 is 
set to zero. 


Appendix G 
Creating Your Own Model or Function Subprogram 


This appendix contains information on the use of subprograms that you can write for both function 
representation in (OF TIMIZE and model representation in (FIT. 


Writing a Model or Function Subprogram 


The purpose of these subprograms is to compute the value of a function and its gradient given certain 
inputs. The inputs vary depending on which program calls your subprogram. If OF TIMIZE calls it, the 
input is an array containing the values of the function’s variables. If CF 1T calls it, the input is two 
arrays, One containing a set of parameters and the other containing data. Because the inputs differ 
between the two programs, there are differences in the subprogram each requires. However, in both 
cases the output is a function value and gradient vector. 


The steps you follow to write a subprogram are the same whether you are writing it for CF IT or 
OPTIMIZE. The steps are as follows: 


e Write the equation describing your function in the form of 
dependent variable = function of independent variables and parameters. 


e If possible, write the partial derivatives with respect to the unknowns. (The unknowns are the 
variables in OF TIMIZE and the parameters in CF IT.) 


e Optionally modify the equations in order to make execution faster. 


e Associate the symbols in your equation with the variables in the standard subprogram syntax for 
your application (either CF IT or OF TIMIZE). 


e Write the subprogram. 
e Design and add any special error checking or other features you want. 


e Check your work and verify your subprogram by using it on a sample problem with known input 
and output values for comparison. 


109 


110 Appendix G: Creating Your Own Model or Function Subprogram 


Standard Subprogram Syntax 


Here is the standard syntax for user subprograms called by any of the programs in this pac. 


For GF TIMIZE: 


SUB nametPoou,Goa,F,03 


Inputs: 
Feo 

Outputs: 
Gea 
F 


For CFIT: 


SUB namec Pov, 863,609, 


Outputs: 


The current value of the variables. 


The gradient of the function at the current variables. 
The function value. 
The condition code. 





F,03 


The current value of the model parameters. 


The independent variables. (The dimension must also 
allow for the dependent variable and the weight.) 


The model gradient with respect to the unknown 
parameters. 


The model value. 
The condition code. 
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Speed and Accuracy 


Gradient 


Including the gradient calculation in your subprogram is optional—the main programs can approximate 
the gradient for you. However, if you can include the gradient calculation in your subprogram, you can 
improve performance of your subprogram in terms of both speed and accuracy. For an explanation of 
how the FP Method uses the gradient calculation, refer to “A Word on Gradient,” page 61. 


Note: 1, you write your own subprogram for a trigonometric function and include the gradient 
calculation, you need to make sure that your subprogram agrees with the machine setting of de- 
grees or radians. It’s usually best to declare in your subprogram which mode you want. Also, when 
you get your results, be sure to interpret them according to the mode the machine is in. 


Speed Tricks 


Anything you can do to increase the execution speed of your subprogram is helpful (especially on long, 
time consuming problems) due to the number of times ©F IT and OF TIMIZE call a subprogram in the 
iteration process. A list of a few time-saving tricks follows, and, with experience, you may develop your 
own techniques to extend the list given here. 


*Substitute multiplication for exponentiation when possible (for example, use ffl rather than A®). 


*Examine the equations for F and G() to see if computing all or part of them in a different order 
will reduce the total number of math operations. 


*If a group of operations occurs repeatedly in your subprogram, calculate it once and assign it to 
some intermediate variable, then use that variable in place of the operations. 


Important Interface Assumptions 


Condition Code 


The condition code variable (C) allows a user-written subprogram to indicate that something has gone 
wrong. The programs in this pac always set C equal to zero before calling a user-written subprogram 
and always test it on return. If the condition code is found to be zero, it indicates there were no excep- 
tional events; if it is found to be positive, it indicates an error; and if it is found to be negative, it 
indicates a warning. Refer to appendix B, starting on page 69, for more information on the relationship 
between condition codes and error messages. 
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Option Base and Option Round 


Both CFIT and OPTIMIZE require you to write your subprogram assuming the base option for 
dimensioning arrays is set to 1 and the rounding option is set to OP TIGH ROUND HEAR (the default 
rounding mode). 


What You Should Not Do 


What follows is a short list of items that you should avoid doing in your subprograms. 


You should not change the values of inputs to your subprogram unless you restore the original values 
on exit. 


The gradient approximation routines assume that the point at which the gradient is being approxi- 
mated does not change -through several calls to your subprogram. If a change to the P() or X() 
variables is made within your subprogram, these routines will not return correct results. 


You should not change certain flag values unless you restore the original values on exit. 

Flags 61, 62, and 63 are inputs to some of the computational routines, specifically GRADF, 
GRAOM, CSG, FIT, and FP. Also, flags 57-60 are used by OPTIMIZE and CF IT. (Although flags 
57-63 are used by the curve fitting programs, their original values are restored when the programs 
end.) If changed within your subprogram and not restored prior to exit, they may give you erro- 
neous results. 


You should not make certain recursive calls. 


Recursive calls are calls to subprograms with passed parameters that will cause it, or something it 
calls, to call your subprogram. 


Some of the binary subprograms in the file FITLIB are not recursive since they use a dedicated 
buffer for intermediate computations. These are GR-ADF, GRAOM, Cou, FF, and FIT. For exam- 
ple, it is acceptable for your subprogram to call FUL‘ to evaluate a polynomial, but your sub- 


program (M'yv SUE) in your file (MF ILE) should not execute the call - 

CALL GRAOFSC"MYSUB", "MYFILE",Poo,Geo,F.23 
This recursive situation has no useful value and will eventually result in an insufficient memory 
error as the two programs keep calling each other indefinitely. 
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Example Subprogram for 2F 70M 


The steps to go through in the process of developing a subprogram for (1F TIMI=E are outlined next in 
the form of an example. 


Function 
Fi(a,b) = a2 — 2ab + 2b? 





Gradient 

: _ f 2a—b) 

eee oe. | 

Subprogram 

4G Sub FLCPC),GO3,F,C3 

2G F=PcCis#re L3-2ERe LIER C22 4+2HPC29¥P 623 | Function Walue 
3H GCL =PEC PCL a-PC Ba | Be g-bo 

40 6G ree ERPOSO-Gels I fb-CP tC a-basedhe 
5a ENOSUE 


Variable Usage 


Variable Use 


7 


The current value of the variables (a and b). 


G3 The gradient of F1 at the current variables. 
F The function value. 
C The condition code. 
Comments 


A few comments about the subprogram are listed below: 
e The computation in line 40 saves one multiplication over the more straight-forward alternative 
(GES2=4kP OSs -2HP O13). 
¢ This example does not include error handling within the subprogram. However, even this simple 


subprogram can encounter exceptional conditions with certain arguments (for example, overflow 
and underflow). 


e The function F'1 achieves its minimum value of 0 at a = b = 0. (You might want to try this exam- 
ple with OF TIMIZE.) 
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Example Subprogram for © F 17 


The steps to go through in the process of developing a subprogram for CF IT are outlined next in the 
form of an example. 


Model 
P(V) =Cx VN 


PVN = C describes the pressure-volume relationship of a certain system during heating. The model 
equation was derived from this equation by rewriting it to get the dependent variable alone on the left. 
The independent variable is V, the dependent variable is P, and the values of N and C are model 
parameters to be determined based on data collected for P as a function of V. 


Gradient 
(-N) 
_ [feapec] __[Vv 
we ea | — LN(V) x | 
Subprogram 
1 Sub PRESSURECE Co, Ato, Goo, Foo 
an BCWSaReT ye e-P ceo foo WA eH 
Ba FePc1s#G015 fo Pek O-HS 
46 GOB s=-LHORO LS a EF I -LH GW OEP CUS 
Sea EHOSUE 
Variable Usage 
Variable Use 
Fa The current value of the model parameters (C and N). 


“t¢ 1 The independent variables (V). 

cc The model gradient with respect to the unknown parameters. 
The model value. 

The condition code. 


op CTL oo. 
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Comments 


A few comments about the subprogram are listed below: 


* The order of computation within the subprogram requires only one exponential evaluation, thus 
providing for faster execution. 


* This example does not include error handling within the subprogram (with reasonable data, excep- 
tional conditions are not likely). 


* One approach to minimize the pain of translating your variables into those used by your sub- 
program follows: 


1. Declare variables you want to use within your subprogram. 

2. Assign to your own variables those input values passed in through the parameter list. 
3. Perform the necessary computations using your variables. 

4. Store the results into the output parameters as required. 


For a good example of this approach, refer to the “Big Box” example on page 22. In this example, 
the computations were done with the declared variables W and H. 


Because it differs from the previous subprogram only in the sign of P(2), you could have used the 
subprogram F'OLMEF in the library file 4.0EL 5 for the model. All you would need to do to use it is 
negate the value returned for P(2) when you get the results. Don’t forget the built-in models! 

* Actually, with two reliable observations, (V,,P,) and (Vo, P), this problem can be solved directly 
and has the solution shown here: 


_ In(V,/V,) 


= as N 
in(P,/P,) d C= PV; 


Often, however, a good fit over a range of observations will result in a curve that more accurately 
reflects your entire data set than that obtained by a direct solution using limited data. 


eee eee Rte! 


Appendix H 


File Names Used in This Pac 


This appendix contains a list of the file names used in this pac. All of these files except one can be 


‘copied to main memory, assuming you have enough memory available. (Cur«s¢F it is the only file 


that cannot be copied.) Most of these files are in ROM and, since your HP-71 searches main RAM first 
when it looks for a file name, be sure to not have any files with the following names in main RAM 
when you are using the curve fitting module. 


File Name 
CurveFit 
CFIT 
OPTIMIZE 
MOOELS 
FCEHTCHI 
FITLE 


FITLIE 
CFREY? 
USERKEYS* 
CREE YS* 


File Type 
LEX 
BASIC 
BASIC 
BASIC 
BASIC 


LEX 


BIN 

KEY 
KEY 
KEY 


destroyed when you run CFIT. 


Description 
Contains the pac identifier. 
Contains the program for curve fitting. 
Contains the program for optimizing. 
Contains the BASIC curve fitting models supplied by the pac. 
Evaluates the left tail of the Chi Square density function. 


Contains the keywords KE''HAIT# and EILLEUFF, the message 
table used by the binary subprograms, and a routine that saves 
scratch memory used by the pac when your machine is turned off. 


Contains all binary user-accessible subprograms. 

Contains key definitions for the (FIT array editor. 

Saves previously user-defined keys while the pac is running. 
Contains a copy in RAM of CFEE''Z. 


*A reserved file name created in RAM when {FIT is running. If you already have a file with this name in RAM, it will be 
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Glossary 


AB 





‘bound estimate: A control input to CF IT and OF TIMIZE that estimates a local minimum or maxi- 
mum value for the function to be optimized. This value is used in the line search algorithm, and, if 
chosen appropriately, can improve performance. 


C 





Chi Square (x”): The function whose value is the sum, over all data, of the squares of the weighted 
differences (Y — F)/W of the dependent variables Y and the model F. If the Y’s are normally 
distributed with mean F and variance W?, then Chi Square is x(v) distributed with v (the number 
of data points minus the number of model parameters) degrees of freedom. 


condition code: A value assigned to the condition code variable. 


condition code variable: A variable passed by reference to a subprogram for the purpose of indicat- 
ing to the calling program the nature of any exceptional event encountered during execution of the 
subprogram. 


controls: The set of inputs used by the computational routines called by CFIT and GFTIMIZE. 


converge: The condition in which the graph of the function being optimized is “sufficiently flat” as 
measured by the gradient norm to stop the iteration process. 


‘DE 
a = ee 


Delta: A control input to CF IT, used in gradient approximation. It is described in CF IT as a “con- 
stant” or “percentage.” 


dependent variable: The measured or Y value that depends on the independent variables (X’s) in 
the data set. 


F 
eee 


Fletcher-Powell Method: An optimization algorithm introduced by R. Fletcher and M. J. D. Powell. 
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G 





gradient: Given a function F(p,,Do,...,P,), the gradient of F is a vector-valued function whose value 
at (p;,P,..-,Pz) is the vector having the partial derivatives of F with respect to the variables 
(D1, P9,---,Pp) aS coordinates. 


gradient norm: With a gradient vector (g,,g.,...,8,), the gradient norm is \ @ +2, 4 bg). 
This value appears in CF IT and OF TIMIZE output identified as |r al. 


H 





HPAF format: Hewlett-Packard Application File format. It refers to a standard DATA file format 
used by HP-71 application pacs. 


LJ,K 





independent variable: A variable controlled by the experimenter. It usually represents a variable 
whose value is selected rather than measured. 


iterate: A value produced by an iterative method. 
iteration: The step number in an iterative method (for example, the 12th iteration). 


iterative method: A method that determines a succession of values where each successive value is 
dependent on one or more of the previous values. 


L 





line search: The algorithmic attempt to minimize a function of one or more variables restricted to a 
particular direction (for example, the attempt to minimize Z = X? + Y? + (Y — 3)? along the line 
Y=X+1). 


M,N 





model function: A function F = F(x,,x9,...,%n3 D1»Po.+++»Pp) of n + k variables. The p’s are param- 
eters to be determined to best fit the data. The x’s are the independent variables in the data. Model 
functions are represented in the pac by subprograms. 


oO 





optimize: The attempt to locate critical points of a function—in particular, local maxima and 
minima. 
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P,Q,R 
eee 


parameter: One of the unknown values that determine the model. The task of CF IT is to produce 
the “best” values for the model parameters. 


percentage goodness of fit: A model evaluation aid equal to 100 x (1 — P) where P is the value 
returned by the subprogram F'C:EHTCHI. Under appropriate conditions this value may be used as 
a model rejection criterion. 


S,T,U 


a ee 


scalar: A numeric value. It is used to distinguish numeric values from vectors. 


scratch memory: The memory used by the numeric computation routines (binary subprograms) to 
store arguments and intermediate results during the computations. 


Vv 

eee 

vector: A vector of length k is an ordered list (p,,po,..., Pp). In this pac, the list elements are scalar 
values, 


W,X,Y,Z 

eee 

weight: A value associated with a data point to provide greater or lesser significance to the point in 
the curve fit process. Relatively large weights correspond to low significance. In order for percent- 
age goodness of fit to have statistical significance, the weights should equal the standard deviations 
of the dependent variables. 
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Data 
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entry 
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saving, 17, 30, 44 
weighting, 29, 31, 52 
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File names used in pac, 117 
Fit menu, 18, 37, 46, 51 
Fit procedure, 14, 36, 41 
FIT subprogram, 89-90, 98 
FITLEX, 100, 117 
FITLIB, 17, 44, 71, 87, 97, 101, 117 
Fletcher-Powell Method, 12, 27, 55, 61-62, 75-76, 84 
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Goto command, 33, 35 
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Horner’s method, 93 
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HP-IL connection, 12 
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inf as a weight, 16, 33, 52 
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KEMAIT4, 100 
Keyboard, redefined, 32 
EK ILLBUFF, 72, 100 
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Loading data (Load), 30 

Library, 3, 27, 38, 101 

LIH subprogram, 17, 93 

Line search, 39, 58, 76-77, 84 

Local best fit, 101 

Local minima/maxima, 20, 39, 55, 80, 101 
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Main menu, 14, 28 
Matrix editor. See Editor 
Memory requirements, 98-100 
Menus, 
CFIT, flow chart, 13 
OPTIMIZE, flow chart, 21 
Messages, error and status, 69-74 
Mode! evaluation, 19, 38, 51-53 
Model specification, 17, 37, 44 
MODELS, 101, 117 
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Optimization of a function, 80 
OPTIMIZE, 3, 20, 55 
OPTIMIZE example, 22-26 
Optimize menu, 24, 56 
OPTIMIZE menus (flow chart), 21 
Option base, 87, 112 
Option round, 112 
Output 

for CF IT, 40, 47-49 

for OF TIMIZE, 25-26, 59-61 
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FCEMTCHI, 54, 94-96, 117 
Penalty function, 85 
Percentage goodness of fit, 51, 54 
FOL'Y subprogram, 44, 93 
Printed output. See Output 
Printer connection, 12 
Product information, 67 
Program controls, 24, 38, 44 
Program exit, 20, 28, 53 
Progress report 

for CFIT, 39 

for OF TIMIZE, 58-60 
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Quit key ((Q]), 32, 36, 39 
Redefined keyboard, 32 
Repair services, 65-66 
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Saving data (a=), 17, 30, 44 
Service 
centers European, 65 
centers U.S., 65 
international, 66 
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shipping instructions, 66 
warranty, 66 
Speed tricks, 111 
Status messages, 69-74 
Steepest descent, 62, 75 
Subprograms, 
Creating, 
for CF IT, 109-110 
for JF TIMIZE, 20, 55, 109-110 
Specifying, 
for CF IT, 17, 37, 44 
for CGPFTIMIZE, 20, 23, 56 


Technical assistance, 67 
Testing a function, 57 
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Service, 66 
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as inf, 16, 33, 51 
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